2024-04-23T23:41:46.966 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T23:41:46.966 DEBUG:teuthology.run:Teuthology command: teuthology --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} --archive /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490 --verbose --name teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/orig.config.yaml 2024-04-23T23:41:47.001 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T23:41:47.091 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} email: ceph-qa@ceph.com first_in_suite: false job_id: '7670490' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writeback global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: b98aab6c00f483888c049d0d8e7bc507e115dd86 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b98aab6c00f483888c049d0d8e7bc507e115dd86 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b98aab6c00f483888c049d0d8e7bc507e115dd86 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9514 sha1: b98aab6c00f483888c049d0d8e7bc507e115dd86 sleep_before_teardown: 0 subset: 69/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b98aab6c00f483888c049d0d8e7bc507e115dd86/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b98aab6c00f483888c049d0d8e7bc507e115dd86 targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDXQVB7gmKJXIocyWXs40/B9kaXk2TIHjpzIeXAurFcvcHgPRDFvMXjIjLN6Yd6Vuo//VH0mQbNnb0S0wOIIr8= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFF1oRZV/vpvscwSRHduZO5gUQD7fU5WJVjy6tDju9NIST6HWEofYM5ALXjQsuvKbIL0KOlCgYZbPYfKQS10kL8= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T23:41:47.091 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b98aab6c00f483888c049d0d8e7bc507e115dd86/qa; will attempt to use it 2024-04-23T23:41:47.092 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b98aab6c00f483888c049d0d8e7bc507e115dd86/qa/tasks 2024-04-23T23:41:47.092 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T23:41:47.094 INFO:teuthology.task.internal:Checking packages... 2024-04-23T23:41:47.119 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b98aab6c00f483888c049d0d8e7bc507e115dd86' 2024-04-23T23:41:47.119 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T23:41:47.120 INFO:teuthology.packaging:ref: None 2024-04-23T23:41:47.120 INFO:teuthology.packaging:tag: None 2024-04-23T23:41:47.120 INFO:teuthology.packaging:branch: squid 2024-04-23T23:41:47.120 INFO:teuthology.packaging:sha1: b98aab6c00f483888c049d0d8e7bc507e115dd86 2024-04-23T23:41:47.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-23T23:41:47.416 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2424.gb98aab6c 2024-04-23T23:41:47.417 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T23:41:47.431 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T23:41:47.431 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T23:41:47.464 INFO:teuthology.task.internal:Saving configuration 2024-04-23T23:41:47.480 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T23:41:47.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T23:41:47.510 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 23:36:47.956752', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDXQVB7gmKJXIocyWXs40/B9kaXk2TIHjpzIeXAurFcvcHgPRDFvMXjIjLN6Yd6Vuo//VH0mQbNnb0S0wOIIr8='} 2024-04-23T23:41:47.533 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 23:36:47.958054', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFF1oRZV/vpvscwSRHduZO5gUQD7fU5WJVjy6tDju9NIST6HWEofYM5ALXjQsuvKbIL0KOlCgYZbPYfKQS10kL8='} 2024-04-23T23:41:47.533 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T23:41:47.540 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-23T23:41:47.540 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-23T23:41:47.540 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T23:41:47.609 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb5cd641e50>, signals=[15]) 2024-04-23T23:41:47.609 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T23:41:47.616 INFO:teuthology.task.internal:Opening connections... 2024-04-23T23:41:47.617 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:41:47.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T23:41:47.700 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:41:47.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T23:41:47.780 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T23:41:47.786 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-04-23T23:41:47.803 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-04-23T23:41:47.803 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-04-23T23:41:47.860 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-04-23T23:41:47.861 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T23:41:47.862 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T23:41:47.862 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T23:41:47.862 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-04-23T23:41:47.888 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-04-23T23:41:47.906 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-04-23T23:41:47.906 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-04-23T23:41:47.962 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-04-23T23:41:47.962 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T23:41:47.963 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T23:41:47.964 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T23:41:47.964 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-04-23T23:41:47.983 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T23:41:47.992 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T23:41:47.998 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T23:41:47.999 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-04-23T23:41:48.002 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-04-23T23:41:48.018 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T23:41:48.025 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T23:41:48.025 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-04-23T23:41:48.058 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-04-23T23:41:48.163 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T23:41:48.216 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T23:41:48.230 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-23T23:41:48.230 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-23T23:41:48.231 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T23:41:48.231 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T23:41:48.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:41:48.247 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-04-23T23:41:48.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:41:48.248 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-04-23T23:41:48.303 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-437.el9.x86_64 2024-04-23T23:41:48.304 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-437.el9.x86_64 2024-04-23T23:41:48.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-04-23T23:41:48.310 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-437.el9.x86_64 2024-04-23T23:41:48.311 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.14.0-437.el9.x86_64 2024-04-23T23:41:48.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum install -y kernel 2024-04-23T23:41:50.592 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-23T23:41:52.592 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-23T23:41:58.545 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 958 kB/s | 8.0 MB 00:08 2024-04-23T23:42:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-23T23:42:04.116 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 794 kB/s | 6.2 MB 00:07 2024-04-23T23:42:06.827 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-23T23:42:07.912 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-23T23:42:08.712 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-04-23T23:42:10.980 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-23T23:42:13.053 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-23T23:42:14.923 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T23:42:14.923 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T23:42:15.012 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:42:15.013 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:42:15.013 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:42:15.114 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2024-04-23T23:42:15.615 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 11:42:13 PM UTC. 2024-04-23T23:42:15.743 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:42:15.752 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:42:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:42:15.753 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-04-23T23:42:15.753 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-04-23T23:42:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:Operation aborted. 2024-04-23T23:42:15.814 INFO:teuthology.orchestra.run.smithi092.stdout:Is this ok [y/N]: 2024-04-23T23:42:15.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2024-04-23T23:42:16.305 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 11:42:13 PM UTC. 2024-04-23T23:42:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:42:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:42:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:42:16.423 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-04-23T23:42:16.423 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-04-23T23:42:16.423 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:42:17.736 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64.rpm 245 kB/s | 255 kB 00:01 2024-04-23T23:42:17.736 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:42:17.736 INFO:teuthology.orchestra.run.smithi092.stdout:Total 194 kB/s | 255 kB 00:01 2024-04-23T23:42:17.761 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:42:17.946 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:42:17.946 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:42:17.952 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:42:17.952 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:42:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:42:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T23:42:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T23:42:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T23:42:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T23:42:19.573 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T23:42:19.574 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:42:19.574 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalled: 2024-04-23T23:42:19.574 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T23:42:19.574 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:42:19.574 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:42:19.790 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T23:42:19.877 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T23:42:19.877 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T23:42:19.878 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T23:42:19.878 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-23T23:42:19.878 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T23:42:19.917 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-04-23T23:42:19.937 DEBUG:teuthology.parallel:result is None 2024-04-23T23:42:20.379 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 2.3 MB/s | 21 MB 00:09 2024-04-23T23:42:25.346 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-23T23:42:27.211 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T23:42:27.211 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T23:42:27.306 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:42:27.307 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:42:27.307 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:42:27.410 DEBUG:teuthology.orchestra.run.smithi119:> echo no | sudo yum reinstall kernel || true 2024-04-23T23:42:27.911 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 11:42:25 PM UTC. 2024-04-23T23:42:28.036 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:42:28.047 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:42:28.047 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-04-23T23:42:28.047 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:42:28.047 INFO:teuthology.orchestra.run.smithi119.stdout:Reinstalling: 2024-04-23T23:42:28.048 INFO:teuthology.orchestra.run.smithi119.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T23:42:28.048 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:42:28.048 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:42:28.048 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:42:28.048 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:42:28.048 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 255 k 2024-04-23T23:42:28.049 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 0 2024-04-23T23:42:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:Operation aborted. 2024-04-23T23:42:28.107 INFO:teuthology.orchestra.run.smithi119.stdout:Is this ok [y/N]: 2024-04-23T23:42:28.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum reinstall -y kernel || true 2024-04-23T23:42:28.600 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 11:42:25 PM UTC. 2024-04-23T23:42:28.716 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:Reinstalling: 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 255 k 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 0 2024-04-23T23:42:28.717 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:42:28.965 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-23T23:42:28.965 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:42:28.965 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-23T23:42:28.982 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:42:29.194 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:42:29.194 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:42:29.199 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:42:29.200 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:42:29.340 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:42:29.407 INFO:teuthology.orchestra.run.smithi119.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T23:42:29.408 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T23:42:30.063 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T23:42:30.063 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T23:42:30.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T23:42:30.768 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:42:30.768 INFO:teuthology.orchestra.run.smithi119.stdout:Reinstalled: 2024-04-23T23:42:30.768 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T23:42:30.768 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:42:30.768 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:42:30.952 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T23:42:30.999 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T23:42:30.999 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T23:42:30.999 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T23:42:30.999 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-23T23:42:30.999 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T23:42:31.031 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-04-23T23:42:31.054 DEBUG:teuthology.parallel:result is None 2024-04-23T23:42:31.054 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T23:42:31.065 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T23:42:31.065 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T23:42:31.068 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T23:42:31.109 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T23:42:31.115 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T23:42:31.121 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T23:42:31.121 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T23:42:31.126 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T23:42:31.172 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T23:42:31.178 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T23:42:31.178 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T23:42:31.201 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T23:42:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T23:42:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T23:42:31.245 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T23:42:31.254 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T23:42:31.256 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T23:42:31.263 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T23:42:31.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T23:42:31.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T23:42:31.320 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T23:42:31.329 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T23:42:31.329 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T23:42:31.352 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T23:42:31.374 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T23:42:31.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T23:42:31.508 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T23:42:31.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T23:42:31.634 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:42:31.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T23:42:31.703 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T23:42:31.744 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T23:42:31.791 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T23:42:31.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T23:42:31.911 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:42:31.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T23:42:31.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-04-23T23:42:31.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-04-23T23:42:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T23:42:32.057 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T23:42:32.326 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T23:42:32.334 INFO:teuthology.task.internal:Starting timer... 2024-04-23T23:42:32.334 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T23:42:32.344 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T23:42:32.352 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-23T23:42:32.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-04-23T23:42:32.442 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-04-23T23:42:32.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-04-23T23:42:32.520 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-04-23T23:42:32.522 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T23:42:32.522 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-04-23T23:42:32.543 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-04-23T23:42:32.543 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-04-23T23:42:32.565 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-04-23T23:42:32.565 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive'} 2024-04-23T23:42:32.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T23:42:32.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:42:32.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T23:42:32.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:42:32.635 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T23:42:32.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-04-23T23:42:32.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-04-23T23:42:32.700 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T23:42:32.711 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-23T23:42:32.716 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-23T23:42:32.847 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T23:42:32.862 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T23:42:32.863 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi119.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T23:46:06.096 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com')] 2024-04-23T23:46:06.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-23T23:46:06.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T23:46:06.176 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-04-23T23:46:06.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-23T23:46:06.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-23T23:46:06.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T23:46:06.331 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-04-23T23:46:06.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-23T23:46:06.417 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T23:46:06.429 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T23:46:06.429 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T23:46:06.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T23:46:06.432 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T23:46:06.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T23:46:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T23:46:06.471 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T23:46:06.484 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T23:46:06.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-04-23T23:46:06.497 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T23:46:06.504 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-04-23T23:46:06.517 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T23:46:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-04-23T23:46:06.529 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-04-23T23:46:06.530 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T23:46:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T23:46:06.554 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T23:46:06.582 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-23T23:46:06.585 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T23:46:06.585 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-23T23:46:06.585 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.585 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.585 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.585 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.607 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-04-23T23:46:06.610 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T23:46:06.610 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-04-23T23:46:06.610 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.610 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.610 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.610 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:46:06.611 INFO:teuthology.run_tasks:Running task install... 2024-04-23T23:46:06.620 DEBUG:teuthology.task.install:project ceph 2024-04-23T23:46:06.620 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}} 2024-04-23T23:46:06.621 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'} 2024-04-23T23:46:06.621 INFO:teuthology.task.install:Using flavor: default 2024-04-23T23:46:06.627 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T23:46:06.628 INFO:teuthology.task.install:extra packages: [] 2024-04-23T23:46:06.628 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86', 'tag': None, 'wait_for_package': False} 2024-04-23T23:46:06.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b98aab6c00f483888c049d0d8e7bc507e115dd86 2024-04-23T23:46:06.630 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86', 'tag': None, 'wait_for_package': False} 2024-04-23T23:46:06.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b98aab6c00f483888c049d0d8e7bc507e115dd86 2024-04-23T23:46:06.778 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/ 2024-04-23T23:46:06.779 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2424.gb98aab6c 2024-04-23T23:46:06.849 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/ 2024-04-23T23:46:06.849 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2424.gb98aab6c 2024-04-23T23:46:06.887 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T23:46:06.887 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:46:06.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T23:46:06.912 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T23:46:06.912 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b98aab6c00f483888c049d0d8e7bc507e115dd86/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T23:46:06.958 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/b98aab6c00f483888c049d0d8e7bc507e115dd86/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T23:46:06.958 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:46:06.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T23:46:06.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T23:46:06.982 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T23:46:06.983 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b98aab6c00f483888c049d0d8e7bc507e115dd86/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T23:46:07.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T23:46:07.049 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T23:46:07.112 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-04-23T23:46:07.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-04-23T23:46:07.119 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T23:46:07.182 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-04-23T23:46:07.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-04-23T23:46:07.339 INFO:teuthology.orchestra.run.smithi092.stdout:48 files removed 2024-04-23T23:46:07.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-04-23T23:46:07.407 INFO:teuthology.orchestra.run.smithi119.stdout:48 files removed 2024-04-23T23:46:07.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw 2024-04-23T23:46:07.978 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 234 kB/s | 83 kB 00:00 2024-04-23T23:46:08.122 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 192 kB/s | 83 kB 00:00 2024-04-23T23:46:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 40 kB/s | 11 kB 00:00 2024-04-23T23:46:08.481 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-23T23:46:08.688 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-23T23:46:09.246 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-23T23:46:10.940 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 784 B/s | 1.9 kB 00:02 2024-04-23T23:46:11.859 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-04-23T23:46:26.011 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-04-23T23:46:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-23T23:46:33.347 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-23T23:46:34.975 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-23T23:46:35.057 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 880 kB/s | 19 MB 00:22 2024-04-23T23:46:40.088 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-23T23:46:40.758 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-04-23T23:46:42.247 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:46:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:46:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:46:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:46:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 10 M 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 3.4 M 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 3.1 M 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 5.4 M 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 21 M 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 25 k 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T23:46:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 745 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 501 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 5.3 M 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 45 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 133 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 162 k 2024-04-23T23:46:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 321 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 299 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 99 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout:Install 30 Packages 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-04-23T23:46:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:46:42.253 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 60 M 2024-04-23T23:46:42.253 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi092.stdout:(1/32): ceph-radosgw-19.0.0-2424.gb98aab6c.el9. 7.1 MB/s | 10 MB 00:01 2024-04-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi092.stdout:(2/32): ceph-selinux-19.0.0-2424.gb98aab6c.el9. 762 kB/s | 25 kB 00:00 2024-04-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi092.stdout:(3/32): ceph-base-19.0.0-2424.gb98aab6c.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-04-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi092.stdout:(4/32): libcephfs2-19.0.0-2424.gb98aab6c.el9.x8 4.2 MB/s | 745 kB 00:00 2024-04-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi092.stdout:(5/32): libradosstriper1-19.0.0-2424.gb98aab6c. 3.5 MB/s | 501 kB 00:00 2024-04-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi092.stdout:(6/32): python3-ceph-argparse-19.0.0-2424.gb98a 492 kB/s | 45 kB 00:00 2024-04-23T23:46:44.573 INFO:teuthology.orchestra.run.smithi092.stdout:(7/32): python3-ceph-common-19.0.0-2424.gb98aab 1.2 MB/s | 133 kB 00:00 2024-04-23T23:46:44.807 INFO:teuthology.orchestra.run.smithi092.stdout:(8/32): ceph-common-19.0.0-2424.gb98aab6c.el9.x 9.6 MB/s | 21 MB 00:02 2024-04-23T23:46:44.832 INFO:teuthology.orchestra.run.smithi092.stdout:(9/32): python3-cephfs-19.0.0-2424.gb98aab6c.el 626 kB/s | 162 kB 00:00 2024-04-23T23:46:44.857 INFO:teuthology.orchestra.run.smithi092.stdout:(10/32): python3-rados-19.0.0-2424.gb98aab6c.el 6.3 MB/s | 321 kB 00:00 2024-04-23T23:46:44.933 INFO:teuthology.orchestra.run.smithi092.stdout:(11/32): librgw2-19.0.0-2424.gb98aab6c.el9.x86_ 8.0 MB/s | 5.3 MB 00:00 2024-04-23T23:46:44.958 INFO:teuthology.orchestra.run.smithi092.stdout:(12/32): python3-rgw-19.0.0-2424.gb98aab6c.el9. 989 kB/s | 99 kB 00:00 2024-04-23T23:46:44.983 INFO:teuthology.orchestra.run.smithi092.stdout:(13/32): python3-rbd-19.0.0-2424.gb98aab6c.el9. 1.9 MB/s | 299 kB 00:00 2024-04-23T23:46:45.138 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T23:46:45.609 INFO:teuthology.orchestra.run.smithi092.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-04-23T23:46:45.709 INFO:teuthology.orchestra.run.smithi092.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 138 kB/s | 107 kB 00:00 2024-04-23T23:46:45.751 INFO:teuthology.orchestra.run.smithi092.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 202 kB/s | 160 kB 00:00 2024-04-23T23:46:46.368 INFO:teuthology.orchestra.run.smithi092.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 68 kB/s | 42 kB 00:00 2024-04-23T23:46:46.668 INFO:teuthology.orchestra.run.smithi092.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 305 kB/s | 292 kB 00:00 2024-04-23T23:46:47.051 INFO:teuthology.orchestra.run.smithi092.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 459 kB/s | 662 kB 00:01 2024-04-23T23:46:47.276 INFO:teuthology.orchestra.run.smithi092.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 59 kB/s | 36 kB 00:00 2024-04-23T23:46:47.327 INFO:teuthology.orchestra.run.smithi092.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 230 kB/s | 221 kB 00:00 2024-04-23T23:46:47.685 INFO:teuthology.orchestra.run.smithi092.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-04-23T23:46:47.860 INFO:teuthology.orchestra.run.smithi092.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 555 kB/s | 309 kB 00:00 2024-04-23T23:46:47.969 INFO:teuthology.orchestra.run.smithi092.stdout:(24/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 92 kB/s | 26 kB 00:00 2024-04-23T23:46:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 363 kB/s | 49 kB 00:00 2024-04-23T23:46:48.304 INFO:teuthology.orchestra.run.smithi092.stdout:(26/32): libarrow-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-04-23T23:46:48.329 INFO:teuthology.orchestra.run.smithi092.stdout:(27/32): libunwind-1.6.2-1.el9.x86_64.rpm 187 kB/s | 67 kB 00:00 2024-04-23T23:46:48.354 INFO:teuthology.orchestra.run.smithi092.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-04-23T23:46:48.421 INFO:teuthology.orchestra.run.smithi092.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-23T23:46:48.721 INFO:teuthology.orchestra.run.smithi092.stdout:(30/32): librbd1-19.0.0-2424.gb98aab6c.el9.x86_ 10 MB/s | 3.1 MB 00:00 2024-04-23T23:46:48.863 INFO:teuthology.orchestra.run.smithi092.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-23T23:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout:(32/32): librados2-19.0.0-2424.gb98aab6c.el9.x8 4.9 MB/s | 3.4 MB 00:00 2024-04-23T23:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout:Total 8.8 MB/s | 60 MB 00:06 2024-04-23T23:46:49.137 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:46:49.174 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:46:49.174 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:46:50.110 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-23T23:46:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:46:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T23:46:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-23T23:46:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-23T23:46:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T23:46:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T23:46:51.524 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T23:46:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T23:46:51.901 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-23T23:46:52.011 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 7/34 2024-04-23T23:46:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 7/34 2024-04-23T23:46:52.192 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 8/34 2024-04-23T23:46:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 9/34 2024-04-23T23:46:52.243 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:46:52.245 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:46:52.245 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:46:52.245 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:46:52.245 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:46:52.245 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 10 M 2024-04-23T23:46:52.245 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 3.4 M 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 3.1 M 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 5.4 M 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 21 M 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 25 k 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 745 k 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T23:46:52.246 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 501 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 5.3 M 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 45 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 133 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 162 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 321 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 299 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 99 k 2024-04-23T23:46:52.247 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout:Install 30 Packages 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 2 Packages 2024-04-23T23:46:52.248 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:46:52.249 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 60 M 2024-04-23T23:46:52.249 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:46:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 9/34 2024-04-23T23:46:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-23T23:46:52.349 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-2424.gb98aab6c.el9. 11/34 2024-04-23T23:46:52.419 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 12/34 2024-04-23T23:46:52.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_6 13/34 2024-04-23T23:46:52.505 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_6 13/34 2024-04-23T23:46:52.540 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-23T23:46:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-23T23:46:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-23T23:46:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-23T23:46:52.747 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-23T23:46:52.776 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-23T23:46:52.809 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-23T23:46:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-23T23:46:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 22/34 2024-04-23T23:46:53.229 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 22/34 2024-04-23T23:46:53.260 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 23/34 2024-04-23T23:46:53.296 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T23:46:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:(1/32): ceph-radosgw-19.0.0-2424.gb98aab6c.el9. 8.5 MB/s | 10 MB 00:01 2024-04-23T23:46:53.753 INFO:teuthology.orchestra.run.smithi119.stdout:(2/32): ceph-base-19.0.0-2424.gb98aab6c.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-04-23T23:46:53.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T23:46:53.778 INFO:teuthology.orchestra.run.smithi119.stdout:(3/32): ceph-selinux-19.0.0-2424.gb98aab6c.el9. 232 kB/s | 25 kB 00:00 2024-04-23T23:46:53.799 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 26/34 2024-04-23T23:46:53.862 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 26/34 2024-04-23T23:46:53.878 INFO:teuthology.orchestra.run.smithi119.stdout:(4/32): libradosstriper1-19.0.0-2424.gb98aab6c. 4.9 MB/s | 501 kB 00:00 2024-04-23T23:46:53.912 INFO:teuthology.orchestra.run.smithi119.stdout:(5/32): libcephfs2-19.0.0-2424.gb98aab6c.el9.x8 4.6 MB/s | 745 kB 00:00 2024-04-23T23:46:53.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 27/34 2024-04-23T23:46:53.960 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-2424.gb98aab6c.el9.x8 28/34 2024-04-23T23:46:53.979 INFO:teuthology.orchestra.run.smithi119.stdout:(6/32): python3-ceph-argparse-19.0.0-2424.gb98a 676 kB/s | 45 kB 00:00 2024-04-23T23:46:54.063 INFO:teuthology.orchestra.run.smithi119.stdout:(7/32): python3-ceph-common-19.0.0-2424.gb98aab 1.6 MB/s | 133 kB 00:00 2024-04-23T23:46:54.155 INFO:teuthology.orchestra.run.smithi119.stdout:(8/32): python3-cephfs-19.0.0-2424.gb98aab6c.el 1.7 MB/s | 162 kB 00:00 2024-04-23T23:46:54.263 INFO:teuthology.orchestra.run.smithi119.stdout:(9/32): python3-rados-19.0.0-2424.gb98aab6c.el9 2.9 MB/s | 321 kB 00:00 2024-04-23T23:46:54.347 INFO:teuthology.orchestra.run.smithi119.stdout:(10/32): librgw2-19.0.0-2424.gb98aab6c.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-04-23T23:46:54.372 INFO:teuthology.orchestra.run.smithi119.stdout:(11/32): python3-rbd-19.0.0-2424.gb98aab6c.el9. 2.7 MB/s | 299 kB 00:00 2024-04-23T23:46:54.397 INFO:teuthology.orchestra.run.smithi119.stdout:(12/32): python3-rgw-19.0.0-2424.gb98aab6c.el9. 1.9 MB/s | 99 kB 00:00 2024-04-23T23:46:54.556 INFO:teuthology.orchestra.run.smithi119.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 583 kB/s | 107 kB 00:00 2024-04-23T23:46:54.773 INFO:teuthology.orchestra.run.smithi119.stdout:(14/32): ceph-common-19.0.0-2424.gb98aab6c.el9. 9.1 MB/s | 21 MB 00:02 2024-04-23T23:46:54.798 INFO:teuthology.orchestra.run.smithi119.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 400 kB/s | 160 kB 00:00 2024-04-23T23:46:54.823 INFO:teuthology.orchestra.run.smithi119.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 169 kB/s | 45 kB 00:00 2024-04-23T23:46:54.883 INFO:teuthology.orchestra.run.smithi119.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 722 kB/s | 42 kB 00:00 2024-04-23T23:46:55.024 INFO:teuthology.orchestra.run.smithi119.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-23T23:46:55.082 INFO:teuthology.orchestra.run.smithi119.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 616 kB/s | 36 kB 00:00 2024-04-23T23:46:55.108 INFO:teuthology.orchestra.run.smithi119.stdout:(20/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 945 kB/s | 292 kB 00:00 2024-04-23T23:46:55.141 INFO:teuthology.orchestra.run.smithi119.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-04-23T23:46:55.176 INFO:teuthology.orchestra.run.smithi119.stdout:(22/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-04-23T23:46:55.301 INFO:teuthology.orchestra.run.smithi119.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-04-23T23:46:55.360 INFO:teuthology.orchestra.run.smithi119.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-04-23T23:46:55.452 INFO:teuthology.orchestra.run.smithi119.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 322 kB/s | 49 kB 00:00 2024-04-23T23:46:55.477 INFO:teuthology.orchestra.run.smithi119.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-23T23:46:55.544 INFO:teuthology.orchestra.run.smithi119.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-23T23:46:55.611 INFO:teuthology.orchestra.run.smithi119.stdout:(28/32): libarrow-9.0.0-10.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-04-23T23:46:55.678 INFO:teuthology.orchestra.run.smithi119.stdout:(29/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-23T23:46:55.719 INFO:teuthology.orchestra.run.smithi119.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-23T23:46:55.995 INFO:teuthology.orchestra.run.smithi119.stdout:(31/32): librbd1-19.0.0-2424.gb98aab6c.el9.x86_ 9.9 MB/s | 3.1 MB 00:00 2024-04-23T23:46:56.128 INFO:teuthology.orchestra.run.smithi119.stdout:(32/32): librados2-19.0.0-2424.gb98aab6c.el9.x8 6.5 MB/s | 3.4 MB 00:00 2024-04-23T23:46:56.130 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:46:56.131 INFO:teuthology.orchestra.run.smithi119.stdout:Total 15 MB/s | 60 MB 00:03 2024-04-23T23:46:56.233 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:46:56.237 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 29/34 2024-04-23T23:46:56.269 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:46:56.269 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:46:56.273 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 29/34 2024-04-23T23:46:56.730 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:46:56.731 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:46:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 29/34 2024-04-23T23:46:56.859 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 30/34 2024-04-23T23:46:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 30/34 2024-04-23T23:46:56.948 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T23:46:56.948 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T23:46:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:46:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 31/34 2024-04-23T23:46:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:46:58.303 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T23:46:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-23T23:46:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-23T23:46:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T23:46:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T23:46:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T23:46:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T23:46:59.073 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-23T23:46:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 7/34 2024-04-23T23:46:59.253 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 7/34 2024-04-23T23:46:59.342 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 8/34 2024-04-23T23:46:59.379 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 9/34 2024-04-23T23:46:59.441 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 9/34 2024-04-23T23:46:59.479 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-23T23:46:59.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-2424.gb98aab6c.el9. 11/34 2024-04-23T23:46:59.601 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 12/34 2024-04-23T23:46:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_6 13/34 2024-04-23T23:46:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_6 13/34 2024-04-23T23:46:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-23T23:46:59.824 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-23T23:46:59.855 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-23T23:46:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-23T23:46:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-23T23:46:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-23T23:46:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-23T23:47:00.293 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-23T23:47:00.354 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 22/34 2024-04-23T23:47:00.444 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 22/34 2024-04-23T23:47:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 23/34 2024-04-23T23:47:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T23:47:00.983 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T23:47:01.015 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 26/34 2024-04-23T23:47:01.085 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 26/34 2024-04-23T23:47:01.155 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 27/34 2024-04-23T23:47:01.192 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-2424.gb98aab6c.el9.x8 28/34 2024-04-23T23:47:03.468 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 29/34 2024-04-23T23:47:03.514 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 29/34 2024-04-23T23:47:04.050 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 29/34 2024-04-23T23:47:04.094 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 30/34 2024-04-23T23:47:04.191 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 30/34 2024-04-23T23:47:04.191 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T23:47:04.191 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T23:47:04.191 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:04.218 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 31/34 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 31/34 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-04-23T23:47:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-04-23T23:47:06.289 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:07.227 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/34 2024-04-23T23:47:07.250 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/34 2024-04-23T23:47:07.250 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:07.251 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T23:47:07.251 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T23:47:07.251 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T23:47:07.251 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:07.252 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T23:47:07.337 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T23:47:07.337 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T23:47:08.483 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T23:47:08.483 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/34 2024-04-23T23:47:08.483 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 5/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_6 6/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 7/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2424.gb98aab6c.el9. 8/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-2424.gb98aab6c.el9.x8 9/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 10/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 11/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 12/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 13/34 2024-04-23T23:47:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T23:47:08.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T23:47:08.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T23:47:08.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T23:47:08.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T23:47:08.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 31/34 2024-04-23T23:47:08.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T23:47:08.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 33/34 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.193 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:09.194 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:09.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-04-23T23:47:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:30 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:10.136 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 48 M 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 163 k 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:47:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:10.138 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-04-23T23:47:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:10.138 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 49 M 2024-04-23T23:47:10.138 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 207 M 2024-04-23T23:47:10.138 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:47:10.446 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): libcephsqlite-19.0.0-2424.gb98aab6c.el9. 689 kB/s | 163 kB 00:00 2024-04-23T23:47:10.485 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-23T23:47:10.611 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 388 kB/s | 64 kB 00:00 2024-04-23T23:47:12.723 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-19.0.0-2424.gb98aab6c.el9.x86_ 19 MB/s | 48 MB 00:02 2024-04-23T23:47:12.725 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:12.725 INFO:teuthology.orchestra.run.smithi092.stdout:Total 19 MB/s | 49 MB 00:02 2024-04-23T23:47:12.752 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:47:12.773 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:47:12.773 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:47:13.138 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:47:13.138 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:47:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 31/34 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-04-23T23:47:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:13.652 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:47:13.749 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-23T23:47:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-23T23:47:13.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/4 2024-04-23T23:47:14.462 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/34 2024-04-23T23:47:14.485 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/34 2024-04-23T23:47:14.486 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:14.486 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T23:47:14.486 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T23:47:14.486 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T23:47:14.486 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:14.488 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T23:47:14.572 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T23:47:14.572 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 5/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_6 6/34 2024-04-23T23:47:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 7/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2424.gb98aab6c.el9. 8/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-2424.gb98aab6c.el9.x8 9/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 10/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 11/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 12/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 13/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T23:47:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 31/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T23:47:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 33/34 2024-04-23T23:47:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T23:47:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:16.449 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:16.715 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-test 2024-04-23T23:47:17.247 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:27 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:17.370 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 48 M 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 163 k 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T23:47:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout:Install 4 Packages 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 49 M 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 207 M 2024-04-23T23:47:17.372 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:47:17.832 INFO:teuthology.orchestra.run.smithi119.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-23T23:47:17.891 INFO:teuthology.orchestra.run.smithi119.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-23T23:47:18.220 INFO:teuthology.orchestra.run.smithi119.stdout:(3/4): libcephsqlite-19.0.0-2424.gb98aab6c.el9. 283 kB/s | 163 kB 00:00 2024-04-23T23:47:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/4 2024-04-23T23:47:18.422 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/4 2024-04-23T23:47:20.803 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/4 2024-04-23T23:47:20.803 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/4 2024-04-23T23:47:20.803 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/4 2024-04-23T23:47:20.803 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-23T23:47:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-23T23:47:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:21.078 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:21.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-04-23T23:47:21.883 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:41 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:22.011 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:22.013 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:47:22.013 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:22.013 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:47:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 6.6 k 2024-04-23T23:47:22.013 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 2.4 M 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 1.5 M 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 249 k 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 4.7 M 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 17 M 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T23:47:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T23:47:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:22.016 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:47:22.017 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:22.017 INFO:teuthology.orchestra.run.smithi092.stdout:Install 36 Packages 2024-04-23T23:47:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:22.017 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 30 M 2024-04-23T23:47:22.017 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 107 M 2024-04-23T23:47:22.017 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:47:22.743 INFO:teuthology.orchestra.run.smithi119.stdout:(4/4): ceph-test-19.0.0-2424.gb98aab6c.el9.x86_ 9.5 MB/s | 48 MB 00:05 2024-04-23T23:47:22.745 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:22.745 INFO:teuthology.orchestra.run.smithi119.stdout:Total 9.1 MB/s | 49 MB 00:05 2024-04-23T23:47:22.772 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:47:22.792 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:47:22.792 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:47:22.823 INFO:teuthology.orchestra.run.smithi092.stdout:(1/36): ceph-19.0.0-2424.gb98aab6c.el9.x86_64.r 40 kB/s | 6.6 kB 00:00 2024-04-23T23:47:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:(2/36): ceph-mgr-19.0.0-2424.gb98aab6c.el9.x86_ 4.3 MB/s | 1.5 MB 00:00 2024-04-23T23:47:23.156 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:47:23.156 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:47:23.349 INFO:teuthology.orchestra.run.smithi092.stdout:(3/36): ceph-mon-19.0.0-2424.gb98aab6c.el9.x86_ 8.9 MB/s | 4.7 MB 00:00 2024-04-23T23:47:23.424 INFO:teuthology.orchestra.run.smithi092.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2424.gb98a 3.2 MB/s | 249 kB 00:00 2024-04-23T23:47:23.591 INFO:teuthology.orchestra.run.smithi092.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-23T23:47:23.671 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:47:23.674 INFO:teuthology.orchestra.run.smithi092.stdout:(6/36): ceph-mds-19.0.0-2424.gb98aab6c.el9.x86_ 2.3 MB/s | 2.4 MB 00:01 2024-04-23T23:47:23.752 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-23T23:47:23.799 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-23T23:47:23.808 INFO:teuthology.orchestra.run.smithi092.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 799 kB/s | 106 kB 00:00 2024-04-23T23:47:23.828 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/4 2024-04-23T23:47:23.975 INFO:teuthology.orchestra.run.smithi092.stdout:(8/36): ceph-osd-19.0.0-2424.gb98aab6c.el9.x86_ 17 MB/s | 17 MB 00:00 2024-04-23T23:47:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-04-23T23:47:24.109 INFO:teuthology.orchestra.run.smithi092.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-04-23T23:47:24.184 INFO:teuthology.orchestra.run.smithi092.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2024-04-23T23:47:24.209 INFO:teuthology.orchestra.run.smithi092.stdout:(12/36): python3-requests-2.25.1-8.el9.noarch.r 534 kB/s | 125 kB 00:00 2024-04-23T23:47:24.535 INFO:teuthology.orchestra.run.smithi092.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 107 kB/s | 35 kB 00:00 2024-04-23T23:47:24.743 INFO:teuthology.orchestra.run.smithi092.stdout:(14/36): lua-5.4.4-4.el9.x86_64.rpm 297 kB/s | 188 kB 00:00 2024-04-23T23:47:24.769 INFO:teuthology.orchestra.run.smithi092.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 295 kB/s | 172 kB 00:00 2024-04-23T23:47:24.794 INFO:teuthology.orchestra.run.smithi092.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 197 kB/s | 51 kB 00:00 2024-04-23T23:47:24.844 INFO:teuthology.orchestra.run.smithi092.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-04-23T23:47:24.919 INFO:teuthology.orchestra.run.smithi092.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-23T23:47:24.944 INFO:teuthology.orchestra.run.smithi092.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-04-23T23:47:25.012 INFO:teuthology.orchestra.run.smithi092.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-23T23:47:25.062 INFO:teuthology.orchestra.run.smithi092.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 214 kB/s | 11 kB 00:00 2024-04-23T23:47:25.087 INFO:teuthology.orchestra.run.smithi092.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.5 MB/s | 358 kB 00:00 2024-04-23T23:47:25.112 INFO:teuthology.orchestra.run.smithi092.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-04-23T23:47:25.138 INFO:teuthology.orchestra.run.smithi092.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-04-23T23:47:25.163 INFO:teuthology.orchestra.run.smithi092.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-04-23T23:47:25.188 INFO:teuthology.orchestra.run.smithi092.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-04-23T23:47:25.247 INFO:teuthology.orchestra.run.smithi092.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 553 kB/s | 46 kB 00:00 2024-04-23T23:47:25.272 INFO:teuthology.orchestra.run.smithi092.stdout:(28/36): lua-devel-5.4.4-4.el9.x86_64.rpm 47 kB/s | 22 kB 00:00 2024-04-23T23:47:25.297 INFO:teuthology.orchestra.run.smithi092.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 724 kB/s | 79 kB 00:00 2024-04-23T23:47:25.339 INFO:teuthology.orchestra.run.smithi092.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-23T23:47:25.364 INFO:teuthology.orchestra.run.smithi092.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-04-23T23:47:25.390 INFO:teuthology.orchestra.run.smithi092.stdout:(32/36): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2024-04-23T23:47:25.415 INFO:teuthology.orchestra.run.smithi092.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-04-23T23:47:25.440 INFO:teuthology.orchestra.run.smithi092.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-23T23:47:25.465 INFO:teuthology.orchestra.run.smithi092.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-04-23T23:47:25.624 INFO:teuthology.orchestra.run.smithi092.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-04-23T23:47:25.625 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:25.625 INFO:teuthology.orchestra.run.smithi092.stdout:Total 8.3 MB/s | 30 MB 00:03 2024-04-23T23:47:25.764 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:47:25.804 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:47:25.804 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:47:26.099 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:47:26.100 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T23:47:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T23:47:27.503 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T23:47:27.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T23:47:27.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T23:47:27.660 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T23:47:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T23:47:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T23:47:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T23:47:27.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T23:47:27.936 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T23:47:27.970 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T23:47:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T23:47:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T23:47:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T23:47:28.160 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T23:47:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T23:47:28.336 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/4 2024-04-23T23:47:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T23:47:28.452 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T23:47:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/4 2024-04-23T23:47:28.513 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T23:47:28.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T23:47:28.601 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T23:47:28.656 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T23:47:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T23:47:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T23:47:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T23:47:29.056 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T23:47:29.105 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T23:47:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T23:47:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2424.gb98aab6c.el9. 31/36 2024-04-23T23:47:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/36 2024-04-23T23:47:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/36 2024-04-23T23:47:29.432 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:29.432 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T23:47:29.432 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T23:47:29.432 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T23:47:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 33/36 2024-04-23T23:47:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 33/36 2024-04-23T23:47:30.892 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:30.892 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T23:47:30.892 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T23:47:30.892 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T23:47:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:30.912 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/4 2024-04-23T23:47:30.912 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/4 2024-04-23T23:47:30.912 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/4 2024-04-23T23:47:30.912 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:31.167 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 34/36 2024-04-23T23:47:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 34/36 2024-04-23T23:47:31.356 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:31.356 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T23:47:31.356 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T23:47:31.356 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T23:47:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:31.461 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph 2024-04-23T23:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 35/36 2024-04-23T23:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 35/36 2024-04-23T23:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T23:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T23:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T23:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:31.679 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 36/36 2024-04-23T23:47:31.969 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:41 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:32.094 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:32.097 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:32.097 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:47:32.097 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:32.097 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:47:32.097 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 6.6 k 2024-04-23T23:47:32.097 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 2.4 M 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 1.5 M 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 249 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 4.7 M 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 17 M 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T23:47:32.098 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T23:47:32.099 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout:Install 36 Packages 2024-04-23T23:47:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:32.101 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 30 M 2024-04-23T23:47:32.101 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 107 M 2024-04-23T23:47:32.101 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:47:33.286 INFO:teuthology.orchestra.run.smithi119.stdout:(1/36): ceph-19.0.0-2424.gb98aab6c.el9.x86_64.r 29 kB/s | 6.6 kB 00:00 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 36/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 5/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2424.gb98aab6c.el9. 6/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T23:47:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T23:47:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T23:47:33.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T23:47:33.612 INFO:teuthology.orchestra.run.smithi119.stdout:(2/36): ceph-mgr-19.0.0-2424.gb98aab6c.el9.x86_ 2.7 MB/s | 1.5 MB 00:00 2024-04-23T23:47:33.745 INFO:teuthology.orchestra.run.smithi119.stdout:(3/36): ceph-mds-19.0.0-2424.gb98aab6c.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-04-23T23:47:33.862 INFO:teuthology.orchestra.run.smithi119.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2424.gb98a 2.1 MB/s | 249 kB 00:00 2024-04-23T23:47:34.021 INFO:teuthology.orchestra.run.smithi119.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-23T23:47:34.146 INFO:teuthology.orchestra.run.smithi119.stdout:(6/36): ceph-mon-19.0.0-2424.gb98aab6c.el9.x86_ 5.5 MB/s | 4.7 MB 00:00 2024-04-23T23:47:34.188 INFO:teuthology.orchestra.run.smithi119.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 7.4 MB/s | 1.2 MB 00:00 2024-04-23T23:47:34.238 INFO:teuthology.orchestra.run.smithi119.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-04-23T23:47:34.288 INFO:teuthology.orchestra.run.smithi119.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-04-23T23:47:34.314 INFO:teuthology.orchestra.run.smithi119.stdout:(10/36): python3-ply-3.11-14.el9.noarch.rpm 636 kB/s | 106 kB 00:00 2024-04-23T23:47:34.339 INFO:teuthology.orchestra.run.smithi119.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-04-23T23:47:34.426 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T23:47:34.427 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T23:47:34.428 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T23:47:34.429 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T23:47:34.430 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T23:47:34.430 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:34.430 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:34.548 INFO:teuthology.orchestra.run.smithi119.stdout:(12/36): ceph-osd-19.0.0-2424.gb98aab6c.el9.x86 18 MB/s | 17 MB 00:00 2024-04-23T23:47:34.623 INFO:teuthology.orchestra.run.smithi119.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 607 kB/s | 172 kB 00:00 2024-04-23T23:47:34.648 INFO:teuthology.orchestra.run.smithi119.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 346 kB/s | 35 kB 00:00 2024-04-23T23:47:34.673 INFO:teuthology.orchestra.run.smithi119.stdout:(15/36): lua-5.4.4-4.el9.x86_64.rpm 524 kB/s | 188 kB 00:00 2024-04-23T23:47:34.699 INFO:teuthology.orchestra.run.smithi119.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 673 kB/s | 51 kB 00:00 2024-04-23T23:47:34.749 INFO:teuthology.orchestra.run.smithi119.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-04-23T23:47:34.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-04-23T23:47:34.891 INFO:teuthology.orchestra.run.smithi119.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 306 kB/s | 43 kB 00:00 2024-04-23T23:47:34.916 INFO:teuthology.orchestra.run.smithi119.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 695 kB/s | 151 kB 00:00 2024-04-23T23:47:34.967 INFO:teuthology.orchestra.run.smithi119.stdout:(20/36): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-04-23T23:47:35.017 INFO:teuthology.orchestra.run.smithi119.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-23T23:47:35.042 INFO:teuthology.orchestra.run.smithi119.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-04-23T23:47:35.068 INFO:teuthology.orchestra.run.smithi119.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2024-04-23T23:47:35.093 INFO:teuthology.orchestra.run.smithi119.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 234 kB/s | 18 kB 00:00 2024-04-23T23:47:35.118 INFO:teuthology.orchestra.run.smithi119.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 306 kB/s | 23 kB 00:00 2024-04-23T23:47:35.143 INFO:teuthology.orchestra.run.smithi119.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2024-04-23T23:47:35.168 INFO:teuthology.orchestra.run.smithi119.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-04-23T23:47:35.194 INFO:teuthology.orchestra.run.smithi119.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-04-23T23:47:35.219 INFO:teuthology.orchestra.run.smithi119.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-04-23T23:47:35.253 INFO:teuthology.orchestra.run.smithi119.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-04-23T23:47:35.278 INFO:teuthology.orchestra.run.smithi119.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-04-23T23:47:35.292 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:55 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:35.311 INFO:teuthology.orchestra.run.smithi119.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 613 kB/s | 36 kB 00:00 2024-04-23T23:47:35.345 INFO:teuthology.orchestra.run.smithi119.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-23T23:47:35.370 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:47:35.370 INFO:teuthology.orchestra.run.smithi119.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-23T23:47:35.395 INFO:teuthology.orchestra.run.smithi119.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-04-23T23:47:35.410 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:35.411 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:47:35.411 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:35.420 INFO:teuthology.orchestra.run.smithi119.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 266 kB/s | 20 kB 00:00 2024-04-23T23:47:35.422 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:35.422 INFO:teuthology.orchestra.run.smithi119.stdout:Total 9.1 MB/s | 30 MB 00:03 2024-04-23T23:47:35.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-04-23T23:47:35.560 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:47:35.599 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:47:35.599 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:47:35.892 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:47:35.894 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:47:35.971 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:55 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:36.088 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 762 k 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:47:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:36.090 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 762 k 2024-04-23T23:47:36.090 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 768 k 2024-04-23T23:47:36.090 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:47:36.516 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-19.0.0-2424.gb98aab6c.el9.noarch.rpm 1.7 MB/s | 762 kB 00:00 2024-04-23T23:47:36.517 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:36.517 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.7 MB/s | 762 kB 00:00 2024-04-23T23:47:36.517 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:47:36.522 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:47:36.522 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:47:36.528 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:47:36.528 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:47:36.595 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:47:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:37.132 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:47:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:37.190 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T23:47:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T23:47:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T23:47:37.323 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T23:47:37.367 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T23:47:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T23:47:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:37.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T23:47:37.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T23:47:37.610 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T23:47:37.646 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T23:47:37.675 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T23:47:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:37.686 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:37.686 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:37.710 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T23:47:37.743 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T23:47:37.784 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T23:47:37.819 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T23:47:37.854 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T23:47:37.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T23:47:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T23:47:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T23:47:38.105 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T23:47:38.192 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T23:47:38.245 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T23:47:38.283 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T23:47:38.332 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T23:47:38.388 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T23:47:38.398 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:58 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 145 k 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:47:38.521 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:38.522 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:47:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:38.522 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 145 k 2024-04-23T23:47:38.522 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 435 k 2024-04-23T23:47:38.522 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:47:38.575 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T23:47:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T23:47:38.700 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T23:47:38.746 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-19.0.0-2424.gb98aab 647 kB/s | 145 kB 00:00 2024-04-23T23:47:38.747 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:38.747 INFO:teuthology.orchestra.run.smithi092.stdout:Total 642 kB/s | 145 kB 00:00 2024-04-23T23:47:38.748 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:47:38.755 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:47:38.755 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:47:38.812 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:47:38.812 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:47:38.812 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T23:47:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T23:47:38.969 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T23:47:38.972 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:47:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c. 1/1 2024-04-23T23:47:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c. 1/1 2024-04-23T23:47:39.072 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:39.072 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T23:47:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2424.gb98aab6c.el9. 31/36 2024-04-23T23:47:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/36 2024-04-23T23:47:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 32/36 2024-04-23T23:47:39.189 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:39.189 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T23:47:39.190 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T23:47:39.190 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T23:47:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:39.936 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c. 1/1 2024-04-23T23:47:39.937 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:39.937 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:39.937 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:39.937 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:39.937 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:40.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-04-23T23:47:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 33/36 2024-04-23T23:47:40.652 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:00 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 33/36 2024-04-23T23:47:40.674 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:40.674 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T23:47:40.674 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T23:47:40.674 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T23:47:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:40.733 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:40.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T23:47:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 34/36 2024-04-23T23:47:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 34/36 2024-04-23T23:47:41.147 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:41.147 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T23:47:41.147 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T23:47:41.147 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T23:47:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:41.344 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:01 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 35/36 2024-04-23T23:47:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 35/36 2024-04-23T23:47:41.389 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:41.389 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T23:47:41.389 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T23:47:41.389 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T23:47:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 36/36 2024-04-23T23:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 3.5 M 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 32 k 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 15 k 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T23:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-04-23T23:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 8.6 M 2024-04-23T23:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 103 M 2024-04-23T23:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:47:41.917 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2424.gb9 168 kB/s | 32 kB 00:00 2024-04-23T23:47:41.942 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2424.gb98 72 kB/s | 15 kB 00:00 2024-04-23T23:47:42.109 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-04-23T23:47:42.176 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): ceph-mgr-dashboard-19.0.0-2424.gb98aab6 7.9 MB/s | 3.5 MB 00:00 2024-04-23T23:47:42.302 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-04-23T23:47:42.368 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.3 MB/s | 863 kB 00:00 2024-04-23T23:47:42.610 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 63 kB/s | 19 kB 00:00 2024-04-23T23:47:42.819 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 854 kB/s | 548 kB 00:00 2024-04-23T23:47:42.944 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 433 kB/s | 144 kB 00:00 2024-04-23T23:47:42.969 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 205 kB/s | 31 kB 00:00 2024-04-23T23:47:43.100 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 36/36 2024-04-23T23:47:43.100 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/36 2024-04-23T23:47:43.100 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/36 2024-04-23T23:47:43.100 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 3/36 2024-04-23T23:47:43.100 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 4/36 2024-04-23T23:47:43.100 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 5/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2424.gb98aab6c.el9. 6/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T23:47:43.101 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T23:47:43.102 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T23:47:43.103 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T23:47:43.104 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T23:47:43.144 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 942 kB/s | 188 kB 00:00 2024-04-23T23:47:43.195 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.5 MB/s | 2.0 MB 00:00 2024-04-23T23:47:43.196 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:43.196 INFO:teuthology.orchestra.run.smithi092.stdout:Total 5.0 MB/s | 8.6 MB 00:01 2024-04-23T23:47:43.244 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:47:43.262 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:47:43.262 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:47:43.415 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:47:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T23:47:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T23:47:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T23:47:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T23:47:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T23:47:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T23:47:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T23:47:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:43.897 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-23T23:47:44.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-23T23:47:44.118 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-23T23:47:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-23T23:47:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-23T23:47:44.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-base 2024-04-23T23:47:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-23T23:47:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-23T23:47:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-23T23:47:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-23T23:47:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2424.gb98aab6c.el9 10/12 2024-04-23T23:47:44.762 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:54 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:44.840 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-base-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:47:44.880 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:44.881 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:47:44.881 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:44.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install cephadm 2024-04-23T23:47:45.455 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:55 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:45.573 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 762 k 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 762 k 2024-04-23T23:47:45.574 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 768 k 2024-04-23T23:47:45.575 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:47:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2424.gb98aab6c.el 11/12 2024-04-23T23:47:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noa 12/12 2024-04-23T23:47:46.125 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm-19.0.0-2424.gb98aab6c.el9.noarch.rpm 1.4 MB/s | 762 kB 00:00 2024-04-23T23:47:46.126 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:46.126 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.3 MB/s | 762 kB 00:00 2024-04-23T23:47:46.126 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:47:46.131 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:47:46.131 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:47:46.137 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:47:46.138 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:47:46.208 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:47:46.626 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noa 12/12 2024-04-23T23:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2424.gb98aab6c.el 1/12 2024-04-23T23:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noa 2/12 2024-04-23T23:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2424.gb98aab6c.el9 3/12 2024-04-23T23:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-23T23:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-23T23:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-23T23:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-23T23:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-23T23:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-23T23:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-23T23:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-23T23:47:47.072 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T23:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T23:47:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T23:47:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:47.169 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/1 2024-04-23T23:47:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:47.278 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:47:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:47.278 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:47.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T23:47:47.489 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T23:47:47.925 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:07 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:47:48.001 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:58 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:48.045 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:47:48.046 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-23T23:47:48.046 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:47:48.046 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-23T23:47:48.046 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 7.4 M 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T23:47:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T23:47:48.048 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T23:47:48.048 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:47:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-23T23:47:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-04-23T23:47:48.048 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-04-23T23:47:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 209 M 2024-04-23T23:47:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:47:48.122 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:48.123 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-04-23T23:47:48.123 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:48.123 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:47:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 145 k 2024-04-23T23:47:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 145 k 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 435 k 2024-04-23T23:47:48.124 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:47:48.340 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-immutable-object-cache-19.0.0-2424.gb98aab 669 kB/s | 145 kB 00:00 2024-04-23T23:47:48.341 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:48.341 INFO:teuthology.orchestra.run.smithi119.stdout:Total 664 kB/s | 145 kB 00:00 2024-04-23T23:47:48.342 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:47:48.349 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:47:48.350 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:47:48.405 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:47:48.405 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:47:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:47:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c. 1/1 2024-04-23T23:47:48.652 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 984 kB/s | 195 kB 00:00 2024-04-23T23:47:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c. 1/1 2024-04-23T23:47:48.657 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:47:48.657 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T23:47:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:48.752 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-04-23T23:47:48.777 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-23T23:47:48.887 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 168 kB/s | 18 kB 00:00 2024-04-23T23:47:48.969 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-04-23T23:47:48.995 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-04-23T23:47:49.137 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2024-04-23T23:47:49.237 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-24 9.5 MB/s | 7.4 MB 00:00 2024-04-23T23:47:49.262 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 913 kB/s | 244 kB 00:00 2024-04-23T23:47:49.362 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-23T23:47:49.454 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.1 MB/s | 250 kB 00:00 2024-04-23T23:47:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c. 1/1 2024-04-23T23:47:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:49.503 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:47:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:47:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:49.503 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:49.725 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr 2024-04-23T23:47:50.231 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:00 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:50.310 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-mgr-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:47:50.352 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:50.353 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:47:50.353 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:50.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T23:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-23T23:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout:Total 15 MB/s | 38 MB 00:02 2024-04-23T23:47:50.949 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:00 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:51.003 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:47:51.029 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:47:51.029 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:47:51.069 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:51.070 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:51.070 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:47:51.070 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 3.5 M 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 32 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 15 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T23:47:51.071 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout:Install 12 Packages 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 8.6 M 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 103 M 2024-04-23T23:47:51.072 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:47:51.437 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:47:51.437 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:47:51.778 INFO:teuthology.orchestra.run.smithi119.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2424.gb98 82 kB/s | 15 kB 00:00 2024-04-23T23:47:51.878 INFO:teuthology.orchestra.run.smithi119.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2424.gb9 111 kB/s | 32 kB 00:00 2024-04-23T23:47:52.045 INFO:teuthology.orchestra.run.smithi119.stdout:(3/12): ceph-mgr-dashboard-19.0.0-2424.gb98aab6 7.8 MB/s | 3.5 MB 00:00 2024-04-23T23:47:52.145 INFO:teuthology.orchestra.run.smithi119.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-04-23T23:47:52.170 INFO:teuthology.orchestra.run.smithi119.stdout:(5/12): python3-protobuf-3.14.0-13.el9.noarch.r 922 kB/s | 269 kB 00:00 2024-04-23T23:47:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:47:52.304 INFO:teuthology.orchestra.run.smithi119.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-23T23:47:52.329 INFO:teuthology.orchestra.run.smithi119.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.0 MB/s | 863 kB 00:00 2024-04-23T23:47:52.342 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T23:47:52.386 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T23:47:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.8 MB/s | 548 kB 00:00 2024-04-23T23:47:52.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T23:47:52.537 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T23:47:52.538 INFO:teuthology.orchestra.run.smithi119.stdout:(9/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 8.7 MB/s | 2.0 MB 00:00 2024-04-23T23:47:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 618 kB/s | 144 kB 00:00 2024-04-23T23:47:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 206 kB/s | 31 kB 00:00 2024-04-23T23:47:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-04-23T23:47:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:Total 5.6 MB/s | 8.6 MB 00:01 2024-04-23T23:47:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:47:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:47:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:47:52.741 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T23:47:52.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T23:47:52.836 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:47:52.836 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:47:52.837 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T23:47:53.361 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:47:53.440 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-23T23:47:53.475 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-23T23:47:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T23:47:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-23T23:47:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-23T23:47:53.589 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T23:47:53.627 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-23T23:47:53.761 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-23T23:47:53.863 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-23T23:47:53.907 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-23T23:47:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-23T23:47:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2424.gb98aab6c.el9 10/12 2024-04-23T23:47:54.805 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T23:47:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2424.gb98aab6c.el 11/12 2024-04-23T23:47:55.035 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noa 12/12 2024-04-23T23:47:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T23:47:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aa 12/12 2024-04-23T23:47:56.404 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noa 12/12 2024-04-23T23:47:56.404 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2424.gb98aab6c.el 1/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noa 2/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2424.gb98aab6c.el9 3/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-23T23:47:56.405 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-23T23:47:56.684 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T23:47:56.685 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:56.686 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:47:56.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T23:47:57.430 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:07 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:47:57.552 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:47:57.553 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-23T23:47:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 7.4 M 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T23:47:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout:Install 12 Packages 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:47:57.555 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 38 M 2024-04-23T23:47:57.556 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 209 M 2024-04-23T23:47:57.556 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:47:58.089 INFO:teuthology.orchestra.run.smithi119.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-23T23:47:58.189 INFO:teuthology.orchestra.run.smithi119.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 330 kB/s | 33 kB 00:00 2024-04-23T23:47:58.215 INFO:teuthology.orchestra.run.smithi119.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-04-23T23:47:58.290 INFO:teuthology.orchestra.run.smithi119.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 242 kB/s | 18 kB 00:00 2024-04-23T23:47:58.349 INFO:teuthology.orchestra.run.smithi119.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-04-23T23:47:58.416 INFO:teuthology.orchestra.run.smithi119.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-04-23T23:47:58.457 INFO:teuthology.orchestra.run.smithi119.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T23:47:58.575 INFO:teuthology.orchestra.run.smithi119.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 5.1 MB 00:00 2024-04-23T23:47:58.642 INFO:teuthology.orchestra.run.smithi119.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2024-04-23T23:47:58.667 INFO:teuthology.orchestra.run.smithi119.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-04-23T23:47:58.676 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aa 12/12 2024-04-23T23:47:58.676 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aa 1/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T23:47:58.677 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T23:47:58.759 INFO:teuthology.orchestra.run.smithi119.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-2 9.4 MB/s | 7.4 MB 00:00 2024-04-23T23:47:58.984 INFO:teuthology.orchestra.run.smithi119.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-04-23T23:47:58.985 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:47:58.985 INFO:teuthology.orchestra.run.smithi119.stdout:Total 27 MB/s | 38 MB 00:01 2024-04-23T23:47:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T23:47:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:59.051 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:47:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:47:59.052 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:47:59.331 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:47:59.357 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:47:59.358 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:47:59.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-04-23T23:47:59.765 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:47:59.766 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:47:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:19 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:00.073 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 49 k 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:48:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:00.075 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-04-23T23:48:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:00.076 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-04-23T23:48:00.076 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-04-23T23:48:00.076 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:00.617 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:00.680 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T23:48:00.724 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T23:48:00.824 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T23:48:00.827 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): ceph-mgr-rook-19.0.0-2424.gb98aab6c.el9 167 kB/s | 49 kB 00:00 2024-04-23T23:48:00.867 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T23:48:01.045 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T23:48:01.052 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 37 kB/s | 19 kB 00:00 2024-04-23T23:48:01.074 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T23:48:01.146 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 42 kB/s | 26 kB 00:00 2024-04-23T23:48:01.150 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T23:48:01.736 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 244 kB/s | 222 kB 00:00 2024-04-23T23:48:01.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T23:48:01.895 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 194 kB/s | 159 kB 00:00 2024-04-23T23:48:01.911 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T23:48:02.062 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 193 kB/s | 32 kB 00:00 2024-04-23T23:48:02.112 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 288 kB/s | 279 kB 00:00 2024-04-23T23:48:02.137 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 199 kB/s | 15 kB 00:00 2024-04-23T23:48:02.379 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-google-auth-2.29.0-1.el9.noarch 771 kB/s | 206 kB 00:00 2024-04-23T23:48:02.396 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-requests-oauthlib-1.3.0-12.el9 81 kB/s | 54 kB 00:00 2024-04-23T23:48:02.429 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 3.5 MB/s | 1.0 MB 00:00 2024-04-23T23:48:02.446 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 891 kB/s | 59 kB 00:00 2024-04-23T23:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 448 kB/s | 90 kB 00:00 2024-04-23T23:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout:Total 894 kB/s | 2.2 MB 00:02 2024-04-23T23:48:02.660 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:02.758 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:02.758 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T23:48:03.255 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T23:48:03.472 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T23:48:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T23:48:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T23:48:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T23:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T23:48:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-23T23:48:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T23:48:04.163 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T23:48:04.191 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aa 12/12 2024-04-23T23:48:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T23:48:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T23:48:04.689 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T23:48:04.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T23:48:04.759 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 13/13 2024-04-23T23:48:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 13/13 2024-04-23T23:48:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 1/13 2024-04-23T23:48:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T23:48:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T23:48:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T23:48:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T23:48:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T23:48:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T23:48:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T23:48:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T23:48:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:05.816 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:06.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T23:48:06.567 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:26 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:06.691 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 141 k 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T23:48:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-04-23T23:48:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-04-23T23:48:06.694 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aa 12/12 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aa 1/12 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T23:48:06.797 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T23:48:06.798 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T23:48:06.798 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T23:48:06.798 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T23:48:06.798 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T23:48:06.798 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T23:48:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T23:48:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T23:48:07.196 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T23:48:07.196 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:07.196 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:07.275 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 914 kB/s | 248 kB 00:00 2024-04-23T23:48:07.292 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2424.gb98aab6c.e 490 kB/s | 141 kB 00:00 2024-04-23T23:48:07.518 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 256 kB/s | 58 kB 00:00 2024-04-23T23:48:07.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-rook 2024-04-23T23:48:07.609 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.8 MB/s | 6.0 MB 00:00 2024-04-23T23:48:07.634 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 448 kB/s | 51 kB 00:00 2024-04-23T23:48:07.693 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-04-23T23:48:07.695 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:07.696 INFO:teuthology.orchestra.run.smithi092.stdout:Total 7.0 MB/s | 7.0 MB 00:01 2024-04-23T23:48:07.766 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:07.875 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:08.061 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:18 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:08.169 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:08.181 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 49 k 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T23:48:08.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:08.184 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:08.185 INFO:teuthology.orchestra.run.smithi119.stdout:Install 13 Packages 2024-04-23T23:48:08.185 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:08.185 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 2.2 M 2024-04-23T23:48:08.185 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 26 M 2024-04-23T23:48:08.185 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T23:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T23:48:08.495 INFO:teuthology.orchestra.run.smithi119.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 239 kB/s | 26 kB 00:00 2024-04-23T23:48:08.520 INFO:teuthology.orchestra.run.smithi119.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 145 kB/s | 19 kB 00:00 2024-04-23T23:48:08.603 INFO:teuthology.orchestra.run.smithi119.stdout:(3/13): ceph-mgr-rook-19.0.0-2424.gb98aab6c.el9 228 kB/s | 49 kB 00:00 2024-04-23T23:48:08.695 INFO:teuthology.orchestra.run.smithi119.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-04-23T23:48:08.721 INFO:teuthology.orchestra.run.smithi119.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 793 kB/s | 159 kB 00:00 2024-04-23T23:48:08.754 INFO:teuthology.orchestra.run.smithi119.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-04-23T23:48:08.838 INFO:teuthology.orchestra.run.smithi119.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-04-23T23:48:08.896 INFO:teuthology.orchestra.run.smithi119.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 183 kB/s | 32 kB 00:00 2024-04-23T23:48:08.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T23:48:08.922 INFO:teuthology.orchestra.run.smithi119.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 90 kB/s | 15 kB 00:00 2024-04-23T23:48:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T23:48:09.053 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T23:48:09.082 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 6/6 2024-04-23T23:48:09.097 INFO:teuthology.orchestra.run.smithi119.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 338 kB/s | 59 kB 00:00 2024-04-23T23:48:09.239 INFO:teuthology.orchestra.run.smithi119.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 634 kB/s | 90 kB 00:00 2024-04-23T23:48:09.489 INFO:teuthology.orchestra.run.smithi119.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 1.7 MB/s | 1.0 MB 00:00 2024-04-23T23:48:09.514 INFO:teuthology.orchestra.run.smithi119.stdout:(13/13): python3-google-auth-2.29.0-1.el9.noarc 304 kB/s | 206 kB 00:00 2024-04-23T23:48:09.515 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:09.515 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-04-23T23:48:09.581 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:09.595 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:09.595 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:09.678 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:09.680 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 6/6 2024-04-23T23:48:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/6 2024-04-23T23:48:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T23:48:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T23:48:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T23:48:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T23:48:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T23:48:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:10.054 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T23:48:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T23:48:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T23:48:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T23:48:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T23:48:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:10.055 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:10.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-04-23T23:48:10.370 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T23:48:10.432 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T23:48:10.487 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T23:48:10.525 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T23:48:10.566 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T23:48:10.661 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T23:48:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-23T23:48:10.785 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:30 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:10.826 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T23:48:10.904 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 889 k 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:10.905 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:10.906 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-04-23T23:48:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:10.906 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 969 k 2024-04-23T23:48:10.906 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.8 M 2024-04-23T23:48:10.906 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:11.178 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 545 kB/s | 80 kB 00:00 2024-04-23T23:48:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-fuse-19.0.0-2424.gb98aab6c.el9.x86_ 2.8 MB/s | 889 kB 00:00 2024-04-23T23:48:11.346 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:11.346 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.1 MB/s | 969 kB 00:00 2024-04-23T23:48:11.358 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:11.380 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:11.380 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:11.457 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:11.458 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:11.539 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T23:48:11.591 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T23:48:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T23:48:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T23:48:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 13/13 2024-04-23T23:48:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T23:48:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/2 2024-04-23T23:48:12.430 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 13/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 1/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T23:48:12.431 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T23:48:12.494 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/2 2024-04-23T23:48:12.494 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/2 2024-04-23T23:48:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T23:48:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:12.739 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T23:48:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:12.739 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T23:48:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:12.743 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:12.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T23:48:12.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-volume 2024-04-23T23:48:13.473 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:23 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:13.505 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:33 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:13.593 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:13.593 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 141 k 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:13.594 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:13.595 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:13.595 INFO:teuthology.orchestra.run.smithi119.stdout:Install 6 Packages 2024-04-23T23:48:13.595 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:13.595 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 7.0 M 2024-04-23T23:48:13.595 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 33 M 2024-04-23T23:48:13.595 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:13.628 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:13.628 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 264 k 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-04-23T23:48:13.629 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:13.630 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 341 k 2024-04-23T23:48:13.630 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 1.5 M 2024-04-23T23:48:13.630 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:13.984 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 365 kB/s | 77 kB 00:00 2024-04-23T23:48:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-volume-19.0.0-2424.gb98aab6c.el9.no 1.1 MB/s | 264 kB 00:00 2024-04-23T23:48:14.010 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:14.010 INFO:teuthology.orchestra.run.smithi092.stdout:Total 895 kB/s | 341 kB 00:00 2024-04-23T23:48:14.017 INFO:teuthology.orchestra.run.smithi119.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-04-23T23:48:14.022 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:14.032 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:14.032 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:14.057 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:14.057 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:14.184 INFO:teuthology.orchestra.run.smithi119.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-23T23:48:14.203 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:14.276 INFO:teuthology.orchestra.run.smithi119.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 631 kB/s | 58 kB 00:00 2024-04-23T23:48:14.302 INFO:teuthology.orchestra.run.smithi119.stdout:(4/6): ceph-mgr-cephadm-19.0.0-2424.gb98aab6c.e 334 kB/s | 141 kB 00:00 2024-04-23T23:48:14.326 INFO:teuthology.orchestra.run.smithi119.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.7 MB/s | 548 kB 00:00 2024-04-23T23:48:14.352 INFO:teuthology.orchestra.run.smithi119.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 676 kB/s | 51 kB 00:00 2024-04-23T23:48:14.353 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:14.353 INFO:teuthology.orchestra.run.smithi119.stdout:Total 9.2 MB/s | 7.0 MB 00:00 2024-04-23T23:48:14.376 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T23:48:14.403 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 2/2 2024-04-23T23:48:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 2/2 2024-04-23T23:48:14.419 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:48:14.419 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T23:48:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:14.424 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:14.436 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:14.436 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:14.532 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:14.533 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:14.832 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:14.991 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T23:48:15.037 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T23:48:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 1/2 2024-04-23T23:48:15.357 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T23:48:15.358 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:15.358 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:15.358 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T23:48:15.358 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:15.552 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T23:48:15.639 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T23:48:15.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-04-23T23:48:15.704 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T23:48:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 6/6 2024-04-23T23:48:16.178 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:36 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:16.303 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 127 k 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 127 k 2024-04-23T23:48:16.304 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 456 k 2024-04-23T23:48:16.305 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 6/6 2024-04-23T23:48:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 1/6 2024-04-23T23:48:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T23:48:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T23:48:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T23:48:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T23:48:16.690 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T23:48:16.690 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:16.691 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:16.916 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-19.0.0-2424.gb98aab6c.el9.x86_64 207 kB/s | 127 kB 00:00 2024-04-23T23:48:16.917 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:16.917 INFO:teuthology.orchestra.run.smithi092.stdout:Total 207 kB/s | 127 kB 00:00 2024-04-23T23:48:16.917 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:16.923 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:16.923 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:16.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-fuse 2024-04-23T23:48:16.988 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:16.988 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:17.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:17.436 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:27 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:17.555 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 889 k 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T23:48:17.556 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout:Install 2 Packages 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 969 k 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 2.8 M 2024-04-23T23:48:17.557 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:17.901 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:17.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:17.901 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:17.901 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:17.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:17.901 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:18.012 INFO:teuthology.orchestra.run.smithi119.stdout:(1/2): ceph-fuse-19.0.0-2424.gb98aab6c.el9.x86_ 3.0 MB/s | 889 kB 00:00 2024-04-23T23:48:18.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-04-23T23:48:18.154 INFO:teuthology.orchestra.run.smithi119.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-04-23T23:48:18.155 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:18.155 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.6 MB/s | 969 kB 00:00 2024-04-23T23:48:18.166 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:18.188 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:18.188 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:18.266 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:18.266 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:18.450 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:18.598 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T23:48:18.634 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:38 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/2 2024-04-23T23:48:18.714 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:18.755 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:18.756 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:18.756 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:18.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-04-23T23:48:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2/2 2024-04-23T23:48:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/2 2024-04-23T23:48:19.337 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:39 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:19.456 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 32 k 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:19.457 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:48:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:19.458 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 32 k 2024-04-23T23:48:19.458 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 142 k 2024-04-23T23:48:19.458 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T23:48:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:19.520 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T23:48:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:19.520 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-19.0.0-2424.gb98aab6c.el9.x86_6 134 kB/s | 32 kB 00:00 2024-04-23T23:48:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:Total 133 kB/s | 32 kB 00:00 2024-04-23T23:48:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:19.699 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:19.699 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:19.701 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:19.701 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:19.784 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:19.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-volume 2024-04-23T23:48:19.890 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:20.250 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:30 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:20.424 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:20.424 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-23T23:48:20.424 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:20.424 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-2424.gb98aab6c.el9 ceph-noarch 264 k 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout:Install 2 Packages 2024-04-23T23:48:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:20.426 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 341 k 2024-04-23T23:48:20.426 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 1.5 M 2024-04-23T23:48:20.426 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:20.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-04-23T23:48:20.843 INFO:teuthology.orchestra.run.smithi119.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 540 kB/s | 77 kB 00:00 2024-04-23T23:48:21.130 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:41 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:21.208 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:21.249 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:21.250 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:21.250 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:21.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-04-23T23:48:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:(2/2): ceph-volume-19.0.0-2424.gb98aab6c.el9.no 410 kB/s | 264 kB 00:00 2024-04-23T23:48:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:Total 371 kB/s | 341 kB 00:00 2024-04-23T23:48:21.356 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:21.365 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:21.365 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:21.393 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:21.393 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T23:48:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 2/2 2024-04-23T23:48:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 2/2 2024-04-23T23:48:21.778 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:48:21.778 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T23:48:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:21.823 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:41 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:21.902 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:21.942 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:21.943 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:21.943 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:22.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-04-23T23:48:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 1/2 2024-04-23T23:48:22.520 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:42 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:22.600 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:22.641 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:22.642 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:22.642 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:22.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-2424.gb98aab6c.el9.noarch 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:22.782 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:23.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados-devel 2024-04-23T23:48:23.210 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:43 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:23.330 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:23.330 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:23.331 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:23.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-04-23T23:48:23.627 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:33 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:23.748 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:23.749 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:23.749 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-04-23T23:48:23.749 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 127 k 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 127 k 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 456 k 2024-04-23T23:48:23.750 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:23.902 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:43 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:23.981 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:24.023 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:24.024 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:24.024 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:24.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-04-23T23:48:24.266 INFO:teuthology.orchestra.run.smithi119.stdout:librados-devel-19.0.0-2424.gb98aab6c.el9.x86_64 246 kB/s | 127 kB 00:00 2024-04-23T23:48:24.267 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:24.267 INFO:teuthology.orchestra.run.smithi119.stdout:Total 245 kB/s | 127 kB 00:00 2024-04-23T23:48:24.267 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:24.273 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:24.273 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:24.336 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:24.596 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:24.604 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:44 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:24.684 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:24.724 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:24.726 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-23T23:48:24.726 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:24.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-04-23T23:48:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:25.249 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:25.249 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:25.249 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:25.249 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:25.249 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:25.249 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:25.312 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:45 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:25.439 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:25.440 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:25.440 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-23T23:48:25.440 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 86 k 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 86 k 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 231 k 2024-04-23T23:48:25.441 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:25.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs2 2024-04-23T23:48:26.005 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:36 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:26.039 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-19.0.0-2424.gb98aab6c.el9.x86_64.rpm 143 kB/s | 86 kB 00:00 2024-04-23T23:48:26.040 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:26.041 INFO:teuthology.orchestra.run.smithi092.stdout:Total 143 kB/s | 86 kB 00:00 2024-04-23T23:48:26.041 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:26.048 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:26.048 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:26.084 INFO:teuthology.orchestra.run.smithi119.stdout:Package libcephfs2-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:26.113 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:26.113 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:26.126 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:26.127 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:26.127 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:26.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs-devel 2024-04-23T23:48:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:26.361 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:26.728 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:36 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:26.741 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:26.847 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:26.848 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:26.848 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-04-23T23:48:26.848 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:26.848 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:26.848 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 32 k 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 32 k 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 142 k 2024-04-23T23:48:26.849 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:26.977 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:26.977 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:27.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-04-23T23:48:27.245 INFO:teuthology.orchestra.run.smithi119.stdout:libcephfs-devel-19.0.0-2424.gb98aab6c.el9.x86_6 80 kB/s | 32 kB 00:00 2024-04-23T23:48:27.245 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:27.246 INFO:teuthology.orchestra.run.smithi119.stdout:Total 80 kB/s | 32 kB 00:00 2024-04-23T23:48:27.246 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:27.249 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:27.249 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:27.252 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:27.252 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:27.343 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:27.726 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:47 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:27.791 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:27.849 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:27.849 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-23T23:48:27.849 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 3.1 M 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:48:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:27.851 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.1 M 2024-04-23T23:48:27.851 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 13 M 2024-04-23T23:48:27.851 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:27.996 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:27.996 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:28.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados2 2024-04-23T23:48:28.693 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:38 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:28.772 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:28.813 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:28.814 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:28.814 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:28.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librbd1 2024-04-23T23:48:29.395 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:39 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:29.473 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:29.514 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:29.515 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:29.515 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:29.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rados 2024-04-23T23:48:30.101 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:40 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:30.180 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rados-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:30.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rgw 2024-04-23T23:48:30.804 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:40 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:30.883 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rgw-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:30.922 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:30.923 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:30.924 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:30.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-cephfs 2024-04-23T23:48:31.309 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-19.0.0-2424.gb98aab6c.el9.x86_64.rpm 915 kB/s | 3.1 MB 00:03 2024-04-23T23:48:31.310 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:31.311 INFO:teuthology.orchestra.run.smithi092.stdout:Total 914 kB/s | 3.1 MB 00:03 2024-04-23T23:48:31.311 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:31.321 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:31.321 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:31.506 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:41 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:31.584 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-cephfs-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:31.625 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:31.626 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:31.626 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:31.692 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rbd 2024-04-23T23:48:31.830 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:31.902 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:31.927 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:48:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T23:48:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T23:48:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T23:48:31.927 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:32.199 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:42 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:32.277 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 is already installed. 2024-04-23T23:48:32.318 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:32.319 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-23T23:48:32.319 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:32.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-fuse 2024-04-23T23:48:32.892 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:32.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:32.893 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:32.893 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:32.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:32.893 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:32.893 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:42 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:33.012 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 86 k 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:48:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:33.014 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 86 k 2024-04-23T23:48:33.014 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 231 k 2024-04-23T23:48:33.014 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:33.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-04-23T23:48:33.598 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-fuse-19.0.0-2424.gb98aab6c.el9.x86_64.rpm 147 kB/s | 86 kB 00:00 2024-04-23T23:48:33.598 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:33.599 INFO:teuthology.orchestra.run.smithi119.stdout:Total 146 kB/s | 86 kB 00:00 2024-04-23T23:48:33.599 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:33.606 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:33.606 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:33.654 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:53 ago on Tue 23 Apr 2024 11:46:40 PM UTC. 2024-04-23T23:48:33.669 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:33.670 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:33.778 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-23T23:48:33.779 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 170 k 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 170 k 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 494 k 2024-04-23T23:48:33.780 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-23T23:48:33.822 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:34.381 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-19.0.0-2424.gb98aab6c.el9.x86_64.rpm 283 kB/s | 170 kB 00:00 2024-04-23T23:48:34.382 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:34.382 INFO:teuthology.orchestra.run.smithi092.stdout:Total 283 kB/s | 170 kB 00:00 2024-04-23T23:48:34.382 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-23T23:48:34.389 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-23T23:48:34.390 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-23T23:48:34.452 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-23T23:48:34.452 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-23T23:48:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:34.531 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:34.531 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:34.603 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-23T23:48:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:34.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-mirror 2024-04-23T23:48:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:35.337 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:45 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:35.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:35.348 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:35.349 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-23T23:48:35.349 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:35.349 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:35.349 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-23T23:48:35.462 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:35.462 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:35.462 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 3.1 M 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 3.1 M 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 13 M 2024-04-23T23:48:35.463 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:35.583 DEBUG:teuthology.parallel:result is None 2024-04-23T23:48:37.710 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-mirror-19.0.0-2424.gb98aab6c.el9.x86_64.rpm 1.4 MB/s | 3.1 MB 00:02 2024-04-23T23:48:37.711 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:37.712 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.4 MB/s | 3.1 MB 00:02 2024-04-23T23:48:37.712 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:37.806 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:37.806 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:38.248 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:38.302 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:38.328 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:38.328 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T23:48:38.328 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T23:48:38.328 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T23:48:38.328 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T23:48:38.328 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:39.372 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:39.372 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:39.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-nbd 2024-04-23T23:48:40.137 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:50 ago on Tue 23 Apr 2024 11:46:50 PM UTC. 2024-04-23T23:48:40.259 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-2424.gb98aab6c.el9 ceph 170 k 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-23T23:48:40.260 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-04-23T23:48:40.261 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-04-23T23:48:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:40.261 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 170 k 2024-04-23T23:48:40.261 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 494 k 2024-04-23T23:48:40.261 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-23T23:48:40.889 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-nbd-19.0.0-2424.gb98aab6c.el9.x86_64.rpm 271 kB/s | 170 kB 00:00 2024-04-23T23:48:40.890 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-23T23:48:40.890 INFO:teuthology.orchestra.run.smithi119.stdout:Total 270 kB/s | 170 kB 00:00 2024-04-23T23:48:40.890 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-23T23:48:40.898 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-23T23:48:40.899 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-23T23:48:40.964 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-23T23:48:40.964 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-23T23:48:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-23T23:48:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 1/1 2024-04-23T23:48:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:41.871 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-23T23:48:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-2424.gb98aab6c.el9.x86_64 2024-04-23T23:48:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:41.872 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-23T23:48:42.132 DEBUG:teuthology.parallel:result is None 2024-04-23T23:48:42.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b98aab6c00f483888c049d0d8e7bc507e115dd86 2024-04-23T23:48:42.350 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T23:48:42.373 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-2424.gb98aab6c.el9 2024-04-23T23:48:42.374 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2424.gb98aab6c.el9 2024-04-23T23:48:42.374 INFO:teuthology.task.install:The correct ceph version 19.0.0-2424.gb98aab6c is installed. 2024-04-23T23:48:42.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b98aab6c00f483888c049d0d8e7bc507e115dd86 2024-04-23T23:48:42.598 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T23:48:42.619 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-2424.gb98aab6c.el9 2024-04-23T23:48:42.620 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2424.gb98aab6c.el9 2024-04-23T23:48:42.620 INFO:teuthology.task.install:The correct ceph version 19.0.0-2424.gb98aab6c is installed. 2024-04-23T23:48:42.622 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T23:48:42.622 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:42.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T23:48:42.649 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:42.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T23:48:42.686 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T23:48:42.686 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:42.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T23:48:42.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T23:48:42.782 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:42.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T23:48:42.806 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T23:48:42.870 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T23:48:42.870 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:42.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T23:48:42.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T23:48:42.960 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:42.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T23:48:42.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T23:48:43.048 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T23:48:43.048 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:43.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T23:48:43.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T23:48:43.138 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:43.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T23:48:43.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T23:48:43.225 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T23:48:43.318 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-23T23:48:43.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-04-23T23:48:43.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-04-23T23:48:43.345 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-23T23:48:43.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T23:48:43.386 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T23:48:43.409 INFO:tasks.ceph:Creating extra log directories... 2024-04-23T23:48:43.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T23:48:43.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T23:48:43.485 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-23T23:48:43.485 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T23:48:43.485 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7670490', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9514, 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86', 'sleep_before_teardown': 0, 'subset': '69/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b98aab6c00f483888c049d0d8e7bc507e115dd86/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86', 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDXQVB7gmKJXIocyWXs40/B9kaXk2TIHjpzIeXAurFcvcHgPRDFvMXjIjLN6Yd6Vuo//VH0mQbNnb0S0wOIIr8=', 'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFF1oRZV/vpvscwSRHduZO5gUQD7fU5WJVjy6tDju9NIST6HWEofYM5ALXjQsuvKbIL0KOlCgYZbPYfKQS10kL8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T23:48:43.486 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T23:48:43.513 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T23:48:43.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T23:48:43.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T23:48:43.592 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:43.592 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T23:48:43.645 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T23:48:43.645 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-23 23:48:32.182871941 +0000 2024-04-23T23:48:43.699 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-23 23:45:47.372804582 +0000 2024-04-23T23:48:43.700 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-23 23:45:47.372804582 +0000 2024-04-23T23:48:43.700 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-23 23:45:47.372804582 +0000 2024-04-23T23:48:43.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T23:48:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-23T23:48:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-23T23:48:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000238913 s, 2.1 MB/s 2024-04-23T23:48:43.763 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T23:48:43.818 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-04-23T23:48:43.873 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T23:48:43.873 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-23 23:48:32.183871913 +0000 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-23 23:45:47.712795078 +0000 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-23 23:45:47.712795078 +0000 2024-04-23T23:48:43.874 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-23 23:45:47.712795078 +0000 2024-04-23T23:48:43.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T23:48:43.935 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-23T23:48:43.936 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-23T23:48:43.936 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00021426 s, 2.4 MB/s 2024-04-23T23:48:43.937 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T23:48:43.993 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-23 23:48:32.183871913 +0000 2024-04-23T23:48:44.048 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-23 23:45:48.042785854 +0000 2024-04-23T23:48:44.049 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-23 23:45:48.042785854 +0000 2024-04-23T23:48:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-23 23:45:48.042785854 +0000 2024-04-23T23:48:44.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T23:48:44.110 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-23T23:48:44.110 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-23T23:48:44.110 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000153456 s, 3.3 MB/s 2024-04-23T23:48:44.111 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T23:48:44.167 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-04-23T23:48:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T23:48:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-23 23:48:32.184871885 +0000 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-23 23:45:48.382776350 +0000 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-23 23:45:48.382776350 +0000 2024-04-23T23:48:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-23 23:45:48.382776350 +0000 2024-04-23T23:48:44.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T23:48:44.284 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-23T23:48:44.284 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-23T23:48:44.284 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000185776 s, 2.8 MB/s 2024-04-23T23:48:44.285 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T23:48:44.341 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T23:48:44.341 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:44.341 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T23:48:44.358 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T23:48:44.358 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-04-23T23:48:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-23 23:48:38.603677846 +0000 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-23 23:45:38.828757399 +0000 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-23 23:45:38.828757399 +0000 2024-04-23T23:48:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-23 23:45:38.828757399 +0000 2024-04-23T23:48:44.415 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T23:48:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-23T23:48:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-23T23:48:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000271149 s, 1.9 MB/s 2024-04-23T23:48:44.505 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T23:48:44.563 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-04-23T23:48:44.619 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T23:48:44.619 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:44.619 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T23:48:44.619 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:44.619 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:44.620 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-23 23:48:38.604677822 +0000 2024-04-23T23:48:44.620 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-23 23:45:39.161749237 +0000 2024-04-23T23:48:44.620 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-23 23:45:39.161749237 +0000 2024-04-23T23:48:44.620 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-23 23:45:39.161749237 +0000 2024-04-23T23:48:44.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T23:48:44.682 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-23T23:48:44.682 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-23T23:48:44.682 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000195581 s, 2.6 MB/s 2024-04-23T23:48:44.683 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T23:48:44.739 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-04-23T23:48:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T23:48:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-23 23:48:38.604677822 +0000 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-23 23:45:39.524740339 +0000 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-23 23:45:39.524740339 +0000 2024-04-23T23:48:44.795 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-23 23:45:39.524740339 +0000 2024-04-23T23:48:44.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T23:48:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-23T23:48:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-23T23:48:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000185784 s, 2.8 MB/s 2024-04-23T23:48:44.858 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T23:48:44.914 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-04-23T23:48:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T23:48:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T23:48:44.970 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-23T23:48:44.970 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T23:48:44.970 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T23:48:44.970 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-23 23:48:38.604677822 +0000 2024-04-23T23:48:44.971 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-23 23:45:39.856732201 +0000 2024-04-23T23:48:44.971 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-23 23:45:39.856732201 +0000 2024-04-23T23:48:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-23 23:45:39.856732201 +0000 2024-04-23T23:48:44.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T23:48:45.032 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-23T23:48:45.032 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-23T23:48:45.032 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000167349 s, 3.1 MB/s 2024-04-23T23:48:45.033 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T23:48:45.089 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T23:48:45.089 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-23T23:48:45.089 INFO:tasks.ceph:Generating config... 2024-04-23T23:48:45.090 INFO:tasks.ceph:[client] rbd cache = True 2024-04-23T23:48:45.090 INFO:tasks.ceph:[client] rbd cache policy = writeback 2024-04-23T23:48:45.090 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-23T23:48:45.090 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-23T23:48:45.090 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-23T23:48:45.091 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T23:48:45.091 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T23:48:45.091 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T23:48:45.091 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T23:48:45.091 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-23T23:48:45.091 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T23:48:45.092 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-23T23:48:45.092 INFO:tasks.ceph:Setting up mon.a... 2024-04-23T23:48:45.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-23T23:48:45.143 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2024-04-23T23:48:45.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-23T23:48:45.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T23:48:45.250 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.c', '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]'), ('mon.b', '172.21.15.119')] 2024-04-23T23:48:45.251 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.119', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writeback'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-23T23:48:45.251 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:45.252 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T23:48:45.305 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --addv c '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]' --add b 172.21.15.119 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 6a51c05a-822c-4999-80c7-e8408cd2a202 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = quincy 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 6a51c05a-822c-4999-80c7-e8408cd2a202 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2024-04-23T23:48:45.404333+0000 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-04-23T23:48:45.404333+0000 2024-04-23T23:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 17 (quincy) 2024-04-23T23:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2024-04-23T23:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-04-23T23:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-23T23:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] mon.c 2024-04-23T23:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-23T23:48:45.392 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T23:48:45.407 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6a51c05a-822c-4999-80c7-e8408cd2a202... 2024-04-23T23:48:45.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T23:48:45.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T23:48:45.487 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-04-23T23:48:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.119" 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on pool no app = False 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 5000 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 6a51c05a-822c-4999-80c7-e8408cd2a202 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-04-23T23:48:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-04-23T23:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: osd shutdown pgref assert = True 2024-04-23T23:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-04-23T23:48:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: rbd cache = True 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: rbd cache policy = writeback 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-04-23T23:48:45.497 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-04-23T23:48:45.498 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-04-23T23:48:45.498 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-04-23T23:48:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-04-23T23:48:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T23:48:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-04-23T23:48:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T23:48:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-04-23T23:48:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.119" 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon client directed command retry = 5 2024-04-23T23:48:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on pool no app = False 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: ms inject socket failures = 5000 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 6a51c05a-822c-4999-80c7-e8408cd2a202 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-04-23T23:48:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-04-23T23:48:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: osd blocked scrub grace period = 3600 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: osd max markdown count = 1000 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: osd shutdown pgref assert = True 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.505 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-23T23:48:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T23:48:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: rbd cache = True 2024-04-23T23:48:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: rbd cache policy = writeback 2024-04-23T23:48:45.509 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-04-23T23:48:45.509 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-04-23T23:48:45.509 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-04-23T23:48:45.510 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-23T23:48:45.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-23T23:48:45.601 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T23:48:45.602 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:45.602 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-23T23:48:45.618 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:45.619 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-23T23:48:45.673 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:48:45.673 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:45.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T23:48:45.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T23:48:45.745 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:48:45.745 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:45.799 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:48:45.799 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:45.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T23:48:45.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T23:48:45.833 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:48:45.833 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:45.888 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T23:48:45.888 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T23:48:45.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-23T23:48:45.937 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-23T23:48:45.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T23:48:45.997 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T23:48:46.000 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T23:48:46.000 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T23:48:46.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T23:48:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-23T23:48:46.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-23T23:48:46.089 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-23T23:48:46.101 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T23:48:46.101 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-23T23:48:46.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-23T23:48:46.124 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T23:48:46.124 INFO:tasks.ceph:role: osd.0 2024-04-23T23:48:46.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:48:46.125 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:48:46.195 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:48:46.196 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:48:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:48:46.196 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:48:46.696 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-23T23:48:46.698 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-23T23:48:46.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-23T23:48:46.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-23T23:48:46.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-23T23:48:46.934 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T23:48:46.934 INFO:tasks.ceph:role: osd.1 2024-04-23T23:48:46.935 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:48:46.935 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:48:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:48:47.461 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-23T23:48:47.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-23T23:48:47.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-23T23:48:47.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-23T23:48:47.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-23T23:48:47.641 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T23:48:47.641 INFO:tasks.ceph:role: osd.2 2024-04-23T23:48:47.641 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:48:47.642 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:48:47.713 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:48:47.714 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:48:47.714 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:48:47.714 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:48:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-23T23:48:48.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-23T23:48:48.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-23T23:48:48.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-23T23:48:48.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-23T23:48:48.420 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T23:48:48.420 INFO:tasks.ceph:role: osd.3 2024-04-23T23:48:48.421 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:48:48.421 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-23T23:48:48.491 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:48:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:48:48.492 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:48:48.979 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-23T23:48:48.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-23T23:48:48.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-23T23:48:49.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-23T23:48:49.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:49.248+0000 7ff853a63640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-23T23:48:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:49.248+0000 7ff853a63640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-23T23:48:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:49.248+0000 7ff853a63640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T23:48:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:49.248+0000 7ff853a63640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T23:48:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:49.248+0000 7ff853a63640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T23:48:49.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:49.249+0000 7ff853a63640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-23T23:48:52.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T23:48:52.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:52.220+0000 7efc7b4a0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-23T23:48:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:52.220+0000 7efc7b4a0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-23T23:48:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:52.220+0000 7efc7b4a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T23:48:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:52.220+0000 7efc7b4a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T23:48:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:52.220+0000 7efc7b4a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T23:48:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:52.221+0000 7efc7b4a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-23T23:48:55.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T23:48:55.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:55.190+0000 7ffb63e63640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-23T23:48:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:55.191+0000 7ffb63e63640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-23T23:48:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:55.191+0000 7ffb63e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T23:48:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:55.191+0000 7ffb63e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T23:48:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:55.191+0000 7ffb63e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T23:48:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:55.191+0000 7ffb63e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-23T23:48:58.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T23:48:58.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:48:58.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:58.167+0000 7f8c2d015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-23T23:48:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:58.168+0000 7f8c2d015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-23T23:48:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:58.168+0000 7f8c2d015640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T23:48:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:58.168+0000 7f8c2d015640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T23:48:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:58.168+0000 7f8c2d015640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T23:48:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-23T23:48:58.168+0000 7f8c2d015640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-23T23:49:00.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T23:49:01.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-23T23:49:01.055 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T23:49:01.055 INFO:tasks.ceph:role: osd.4 2024-04-23T23:49:01.055 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:49:01.055 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T23:49:01.129 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:49:01.129 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:49:01.130 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:49:01.130 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:49:01.130 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:49:01.130 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:49:01.130 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:49:01.131 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:49:01.131 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:49:01.131 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:49:01.627 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-23T23:49:01.629 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-23T23:49:01.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-23T23:49:01.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-23T23:49:01.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-23T23:49:01.836 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T23:49:01.836 INFO:tasks.ceph:role: osd.5 2024-04-23T23:49:01.836 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:49:01.836 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:49:01.909 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:49:01.910 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:49:01.910 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:49:02.364 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-23T23:49:02.366 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-23T23:49:02.367 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-23T23:49:02.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-23T23:49:02.510 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-23T23:49:02.575 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T23:49:02.575 INFO:tasks.ceph:role: osd.6 2024-04-23T23:49:02.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:49:02.575 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T23:49:02.645 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:49:02.646 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:49:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:49:02.647 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:49:03.144 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-23T23:49:03.146 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-23T23:49:03.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-23T23:49:03.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-23T23:49:03.271 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-23T23:49:03.335 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T23:49:03.335 INFO:tasks.ceph:role: osd.7 2024-04-23T23:49:03.335 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:49:03.335 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-23T23:49:03.405 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T23:49:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T23:49:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T23:49:03.405 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T23:49:03.406 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T23:49:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-23T23:49:03.406 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T23:49:03.406 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T23:49:03.406 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T23:49:03.406 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T23:49:03.914 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-23T23:49:03.916 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-23T23:49:03.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-23T23:49:04.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-23T23:49:04.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:49:04.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:04.168+0000 7fd4648b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-23T23:49:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:04.169+0000 7fd4648b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-23T23:49:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:04.169+0000 7fd4648b5640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T23:49:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:04.169+0000 7fd4648b5640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T23:49:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:04.169+0000 7fd4648b5640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T23:49:04.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:04.176+0000 7fd4648b5640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-23T23:49:07.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-23T23:49:07.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:49:07.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:07.138+0000 7fe25dad5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-23T23:49:07.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:07.138+0000 7fe25dad5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-23T23:49:07.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:07.138+0000 7fe25dad5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T23:49:07.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:07.138+0000 7fe25dad5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T23:49:07.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:07.138+0000 7fe25dad5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T23:49:07.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:07.139+0000 7fe25dad5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-23T23:49:09.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-23T23:49:10.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:49:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:10.111+0000 7f1d6ce18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-23T23:49:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:10.112+0000 7f1d6ce18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-23T23:49:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:10.112+0000 7f1d6ce18640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T23:49:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:10.112+0000 7f1d6ce18640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T23:49:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:10.112+0000 7f1d6ce18640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T23:49:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:10.112+0000 7f1d6ce18640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-23T23:49:12.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-23T23:49:12.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:49:13.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:13.086+0000 7f5251e07640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-23T23:49:13.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:13.086+0000 7f5251e07640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-23T23:49:13.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:13.086+0000 7f5251e07640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T23:49:13.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:13.086+0000 7f5251e07640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T23:49:13.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:13.086+0000 7f5251e07640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T23:49:13.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-23T23:49:13.087+0000 7f5251e07640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-23T23:49:15.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-23T23:49:15.955 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T23:49:15.955 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:15.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-23T23:49:15.983 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:15.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-23T23:49:16.051 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:16.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-23T23:49:16.119 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:16.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-23T23:49:16.187 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:16.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-23T23:49:16.255 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:16.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-23T23:49:16.283 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:16.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-23T23:49:16.350 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:16.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-23T23:49:16.415 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:16.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-23T23:49:16.481 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:16.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-23T23:49:16.546 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:16.546 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-23T23:49:16.563 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:16.563 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-23T23:49:16.602 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T23:49:16.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T23:49:16.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T23:49:16.646 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2024-04-23T23:49:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDdSChmJLq+OBAAXm5mk7b9NbdG2/mDiJJY6Q== 2024-04-23T23:49:16.646 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-04-23T23:49:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDhSChmEd7dDhAATub4eojadxLUCQgyazj68w== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDkSChmNisxDRAARuGt6ar0viBvmBkIyEXBCg== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDnSChm/rpuCxAAzL///+gtV6wVHqxJzv1v5A== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDqSChmu4UPChAAgjoAIl40+hOQ8ic4HTXzCg== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDeSChma7G8ABAAbEx42N9lEFZ8YyjtiVKWpQ== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDwSChmHMcWChAAMBAGAIN5m0bQcs5DGQDc9g== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDzSChmvTlbCBAA0a5C3d+U1hRXd5/l2Ihf7w== 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-04-23T23:49:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD2SChmYGTHBhAA+6vKSTy1pNzQegHHlyx5nw== 2024-04-23T23:49:16.648 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-04-23T23:49:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD5SChmtfMwBRAAUHSqWZSw4+HwDIgvsJ5D+A== 2024-04-23T23:49:16.648 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-04-23T23:49:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDeSChmuXY0AxAAyARgmiH1fzlXfSNE45QIfA== 2024-04-23T23:49:16.648 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2024-04-23T23:49:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDeSChmBtw1BhAASfmliyQDuTGAyeu+dy1iGQ== 2024-04-23T23:49:16.666 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2024-04-23T23:49:16.666 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDdSChmJLq+OBAAXm5mk7b9NbdG2/mDiJJY6Q== 2024-04-23T23:49:16.666 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-04-23T23:49:16.666 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDhSChmEd7dDhAATub4eojadxLUCQgyazj68w== 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDkSChmNisxDRAARuGt6ar0viBvmBkIyEXBCg== 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDnSChm/rpuCxAAzL///+gtV6wVHqxJzv1v5A== 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDqSChmu4UPChAAgjoAIl40+hOQ8ic4HTXzCg== 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDeSChma7G8ABAAbEx42N9lEFZ8YyjtiVKWpQ== 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDwSChmHMcWChAAMBAGAIN5m0bQcs5DGQDc9g== 2024-04-23T23:49:16.667 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDzSChmvTlbCBAA0a5C3d+U1hRXd5/l2Ihf7w== 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD2SChmYGTHBhAA+6vKSTy1pNzQegHHlyx5nw== 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQD5SChmtfMwBRAAUHSqWZSw4+HwDIgvsJ5D+A== 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDeSChmuXY0AxAAyARgmiH1fzlXfSNE45QIfA== 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2024-04-23T23:49:16.668 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDeSChmBtw1BhAASfmliyQDuTGAyeu+dy1iGQ== 2024-04-23T23:49:16.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T23:49:16.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T23:49:16.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:16.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T23:49:16.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T23:49:17.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.014 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T23:49:17.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T23:49:17.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T23:49:17.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T23:49:17.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T23:49:17.320 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T23:49:17.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-23T23:49:17.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T23:49:17.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-23T23:49:17.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-23T23:49:17.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T23:49:17.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-23T23:49:17.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-23T23:49:17.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T23:49:17.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-23T23:49:17.782 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:49:17.784 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T23:49:17.837 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-23T23:49:17.837 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-23T23:49:17.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-23T23:49:17.840 INFO:tasks.ceph.mon.a:Started 2024-04-23T23:49:17.840 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-23T23:49:17.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-23T23:49:17.842 INFO:tasks.ceph.mon.c:Started 2024-04-23T23:49:17.842 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-23T23:49:17.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-23T23:49:17.880 INFO:tasks.ceph.mon.b:Started 2024-04-23T23:49:17.880 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-23T23:49:17.880 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-23T23:49:17.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-23T23:49:17.883 INFO:tasks.ceph.mgr.y:Started 2024-04-23T23:49:17.883 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-23T23:49:17.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-23T23:49:17.885 INFO:tasks.ceph.mgr.x:Started 2024-04-23T23:49:17.885 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T23:49:17.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2024-04-23T23:49:23.273 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:23.287+0000 7f068ee52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T23:49:23.427 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:23.440+0000 7f068ee52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T23:49:23.480 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:23.494+0000 7f068ee52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T23:49:23.752 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:23.766+0000 7f068ee52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T23:49:23.799 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:23.812+0000 7f068ee52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T23:49:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-04-23T23:49:24.077 INFO:tasks.ceph:check_enable_crimson: False 2024-04-23T23:49:24.077 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-23T23:49:24.077 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:24.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-23T23:49:24.081 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:24.095+0000 7fd550ab6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T23:49:24.102 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:24.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-23T23:49:24.166 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:24.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-23T23:49:24.230 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-23T23:49:24.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-23T23:49:24.236 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:24.250+0000 7fd550ab6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T23:49:24.259 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:24.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-23T23:49:24.282 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:24.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-23T23:49:24.292 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:24.306+0000 7fd550ab6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T23:49:24.346 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:24.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-23T23:49:24.410 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-23T23:49:24.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-23T23:49:24.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 6a84eb3e-7d8a-4436-956d-f7da68d06a21 0 2024-04-23T23:49:24.535 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:24.549+0000 7f068ee52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T23:49:24.538 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:24.552+0000 7fd550ab6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T23:49:24.583 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:24.596+0000 7fd550ab6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T23:49:24.626 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:24.639+0000 7f068ee52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T23:49:24.670 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:24.684+0000 7f068ee52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T23:49:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:0 2024-04-23T23:49:24.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new be35c480-9a84-4846-9f0b-043c7c2c9f4a 1 2024-04-23T23:49:24.845 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:24.858+0000 7f068ee52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T23:49:24.900 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:24.914+0000 7f068ee52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T23:49:24.929 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-23T23:49:24.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 64e8bbbb-690a-4267-a030-30ec66f30bfa 2 2024-04-23T23:49:24.956 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:24.969+0000 7f068ee52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T23:49:25.074 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:25.088+0000 7f068ee52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T23:49:25.124 INFO:teuthology.orchestra.run.smithi119.stdout:2 2024-04-23T23:49:25.135 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 692beda9-1302-4d96-81ab-62c40a12e3f1 3 2024-04-23T23:49:25.276 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.290+0000 7fd550ab6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T23:49:25.360 INFO:teuthology.orchestra.run.smithi119.stdout:3 2024-04-23T23:49:25.363 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.377+0000 7fd550ab6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T23:49:25.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 705f6b1f-c35d-4ed8-a2c9-70b9324691e3 4 2024-04-23T23:49:25.405 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.419+0000 7fd550ab6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T23:49:25.474 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:25.487+0000 7f068ee52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T23:49:25.555 INFO:teuthology.orchestra.run.smithi119.stdout:4 2024-04-23T23:49:25.566 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new eb37446d-b5c9-4200-a1e9-38c1f6268401 5 2024-04-23T23:49:25.584 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.598+0000 7fd550ab6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T23:49:25.632 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.645+0000 7fd550ab6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T23:49:25.684 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.698+0000 7fd550ab6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T23:49:25.744 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:25.758+0000 7f068ee52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T23:49:25.749 INFO:teuthology.orchestra.run.smithi119.stdout:5 2024-04-23T23:49:25.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new c38a4da5-1017-4bb9-86ab-999ea57999c9 6 2024-04-23T23:49:25.795 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:25.809+0000 7fd550ab6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T23:49:25.800 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:25.814+0000 7f068ee52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T23:49:25.923 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:25.936+0000 7f068ee52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T23:49:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:6 2024-04-23T23:49:25.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new a921d465-920a-4ad0-839f-d97cd5e6ce62 7 2024-04-23T23:49:26.173 INFO:teuthology.orchestra.run.smithi119.stdout:7 2024-04-23T23:49:26.177 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:26.191+0000 7fd550ab6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T23:49:26.184 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-23T23:49:26.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-23T23:49:26.187 INFO:tasks.ceph.osd.0:Started 2024-04-23T23:49:26.187 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-23T23:49:26.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-23T23:49:26.189 INFO:tasks.ceph.osd.1:Started 2024-04-23T23:49:26.190 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-23T23:49:26.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-23T23:49:26.192 INFO:tasks.ceph.osd.2:Started 2024-04-23T23:49:26.192 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-23T23:49:26.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-23T23:49:26.195 INFO:tasks.ceph.osd.3:Started 2024-04-23T23:49:26.195 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-23T23:49:26.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-23T23:49:26.197 INFO:tasks.ceph.osd.4:Started 2024-04-23T23:49:26.197 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-23T23:49:26.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-23T23:49:26.199 INFO:tasks.ceph.osd.5:Started 2024-04-23T23:49:26.199 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-23T23:49:26.199 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-23T23:49:26.201 INFO:tasks.ceph.osd.6:Started 2024-04-23T23:49:26.201 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-23T23:49:26.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-23T23:49:26.203 INFO:tasks.ceph.osd.7:Started 2024-04-23T23:49:26.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T23:49:26.216 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.230+0000 7f068ee52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T23:49:26.275 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.288+0000 7f068ee52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T23:49:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:26.389 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:49:26.183177+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T23:49:26.183177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:49:26.400 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-23T23:49:26.400 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T23:49:26.440 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.454+0000 7f068ee52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T23:49:26.470 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:26.484+0000 7fd550ab6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T23:49:26.523 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:26.536+0000 7fd550ab6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T23:49:26.578 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.591+0000 7f068ee52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T23:49:26.627 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.640+0000 7f068ee52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T23:49:26.640 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:26.653+0000 7fd550ab6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T23:49:26.695 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.708+0000 7f068ee52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T23:49:26.740 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:26.754+0000 7f068ee52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T23:49:26.802 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:26.815+0000 7fe7151b8640 -1 Falling back to public interface 2024-04-23T23:49:26.803 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:26.816+0000 7fa30fdf5640 -1 Falling back to public interface 2024-04-23T23:49:26.809 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:26.822+0000 7f581b213640 -1 Falling back to public interface 2024-04-23T23:49:26.813 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:26.827+0000 7f6fc44a0640 -1 Falling back to public interface 2024-04-23T23:49:26.814 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:26.827+0000 7f8f9269a640 -1 Falling back to public interface 2024-04-23T23:49:26.817 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:26.831+0000 7f5a2ec13640 -1 Falling back to public interface 2024-04-23T23:49:26.823 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:26.837+0000 7f7d30a0d640 -1 Falling back to public interface 2024-04-23T23:49:26.824 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:26.837+0000 7fc009cd9640 -1 Falling back to public interface 2024-04-23T23:49:26.918 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:26.932+0000 7fd550ab6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T23:49:26.965 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:26.979+0000 7fd550ab6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T23:49:27.122 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:27.135+0000 7fd550ab6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T23:49:27.205 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T23:49:27.217 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:27.231+0000 7f068ee52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T23:49:27.262 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:27.276+0000 7fd550ab6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T23:49:27.323 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:27.337+0000 7fd550ab6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T23:49:27.334 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:27.347+0000 7f068ee52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T23:49:27.392 INFO:teuthology.misc.health.smithi092.stdout: 2024-04-23T23:49:27.392 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:49:26.183177+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T23:49:26.183177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:49:27.402 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-23T23:49:27.410 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:27.424+0000 7fd550ab6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T23:49:27.456 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:27.470+0000 7fd550ab6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T23:49:27.928 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:27.942+0000 7fd550ab6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T23:49:28.044 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-23T23:49:28.058+0000 7fd550ab6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T23:49:28.348 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:28.361+0000 7f062fb5b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-23T23:49:28.348 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-04-23T23:49:28.362+0000 7f062fb5b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-23T23:49:30.252 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:30.266+0000 7fe7151b8640 -1 osd.5 0 log_to_monitors true 2024-04-23T23:49:30.283 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:30.296+0000 7f581b213640 -1 osd.2 0 log_to_monitors true 2024-04-23T23:49:30.307 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:30.321+0000 7fa30fdf5640 -1 osd.7 0 log_to_monitors true 2024-04-23T23:49:30.315 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:30.329+0000 7f6fc44a0640 -1 osd.6 0 log_to_monitors true 2024-04-23T23:49:30.330 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:30.343+0000 7f5a2ec13640 -1 osd.4 0 log_to_monitors true 2024-04-23T23:49:30.348 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:30.362+0000 7f8f9269a640 -1 osd.3 0 log_to_monitors true 2024-04-23T23:49:30.360 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:30.374+0000 7f7d30a0d640 -1 osd.1 0 log_to_monitors true 2024-04-23T23:49:30.366 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:30.379+0000 7fc009cd9640 -1 osd.0 0 log_to_monitors true 2024-04-23T23:49:32.223 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:32.237+0000 7f58162bd640 -1 osd.2 0 waiting for initial osdmap 2024-04-23T23:49:32.234 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:32.248+0000 7f580e8ef640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:33.410 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:33.423+0000 7fe712163640 -1 osd.5 0 waiting for initial osdmap 2024-04-23T23:49:33.411 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:33.425+0000 7f5a29cbd640 -1 osd.4 0 waiting for initial osdmap 2024-04-23T23:49:33.418 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:33.432+0000 7fa30b568640 -1 osd.7 0 waiting for initial osdmap 2024-04-23T23:49:33.421 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:33.435+0000 7fe708f5d640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:33.426 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:33.440+0000 7f5a222ef640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:33.429 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:33.443+0000 7fa303b9a640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:33.456 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:33.469+0000 7f6fc144b640 -1 osd.6 0 waiting for initial osdmap 2024-04-23T23:49:33.465 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:33.479+0000 7f6fb8245640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:34.163 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:34.176+0000 7f8f8de0d640 -1 osd.3 0 waiting for initial osdmap 2024-04-23T23:49:34.181 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:34.195+0000 7f8f8643f640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:34.209 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T23:49:34.212 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:34.225+0000 7f7d2bb20640 -1 osd.1 0 waiting for initial osdmap 2024-04-23T23:49:34.218 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:34.231+0000 7fc00544c640 -1 osd.0 0 waiting for initial osdmap 2024-04-23T23:49:34.225 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:34.239+0000 7f7d24152640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:34.229 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:34.243+0000 7fbffda7e640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:49:34.485 INFO:teuthology.misc.health.smithi092.stdout: 2024-04-23T23:49:34.485 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":14,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:49:33.611959+0000","last_up_change":"2024-04-23T23:49:33.611959+0000","last_in_change":"2024-04-23T23:49:26.183177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:49:34.496 DEBUG:teuthology.misc:5 of 8 OSDs are up 2024-04-23T23:49:41.304 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T23:49:41.588 INFO:teuthology.misc.health.smithi092.stdout: 2024-04-23T23:49:41.588 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":18,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:49:38.235939+0000","last_up_change":"2024-04-23T23:49:35.179323+0000","last_in_change":"2024-04-23T23:49:26.183177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T23:49:35.387984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6820","nonce":3364976130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6824","nonce":3364976130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6831","nonce":3364976130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6828","nonce":3364976130}]},"public_addr":"172.21.15.92:6820/3364976130","cluster_addr":"172.21.15.92:6824/3364976130","heartbeat_back_addr":"172.21.15.92:6831/3364976130","heartbeat_front_addr":"172.21.15.92:6828/3364976130","state":["exists","up"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6817","nonce":2924722109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6821","nonce":2924722109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6829","nonce":2924722109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6823","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6825","nonce":2924722109}]},"public_addr":"172.21.15.92:6817/2924722109","cluster_addr":"172.21.15.92:6821/2924722109","heartbeat_back_addr":"172.21.15.92:6829/2924722109","heartbeat_front_addr":"172.21.15.92:6825/2924722109","state":["exists","up"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6809","nonce":1273528383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6811","nonce":1273528383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6815","nonce":1273528383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6813","nonce":1273528383}]},"public_addr":"172.21.15.92:6809/1273528383","cluster_addr":"172.21.15.92:6811/1273528383","heartbeat_back_addr":"172.21.15.92:6815/1273528383","heartbeat_front_addr":"172.21.15.92:6813/1273528383","state":["exists","up"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.405502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.334094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.367807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.328752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.255936+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.298943+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.352632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:49:41.599 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-23T23:49:41.600 INFO:tasks.ceph:Creating RBD pool 2024-04-23T23:49:41.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-23T23:49:42.239 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2024-04-23T23:49:42.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-23T23:49:43.238 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-23T23:49:43.254 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-23T23:49:43.254 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T23:49:43.255 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-23T23:49:43.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-23T23:49:43.255 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T23:49:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:43.602 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":97797378},{"type":"v1","addr":"172.21.15.119:6833","nonce":97797378}]},"active_addr":"172.21.15.119:6833/97797378","active_change":"2024-04-23T23:49:27.350407+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":4083113641}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":732867473}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":967147046}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":794247454}]}]} 2024-04-23T23:49:43.608 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-23T23:49:43.608 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T23:49:43.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T23:49:43.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:43.887 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:49:43.250751+0000","last_up_change":"2024-04-23T23:49:35.179323+0000","last_in_change":"2024-04-23T23:49:26.183177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T23:49:35.387984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T23:49:41.909447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6820","nonce":3364976130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6824","nonce":3364976130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6831","nonce":3364976130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6828","nonce":3364976130}]},"public_addr":"172.21.15.92:6820/3364976130","cluster_addr":"172.21.15.92:6824/3364976130","heartbeat_back_addr":"172.21.15.92:6831/3364976130","heartbeat_front_addr":"172.21.15.92:6828/3364976130","state":["exists","up"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6817","nonce":2924722109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6821","nonce":2924722109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6829","nonce":2924722109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6823","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6825","nonce":2924722109}]},"public_addr":"172.21.15.92:6817/2924722109","cluster_addr":"172.21.15.92:6821/2924722109","heartbeat_back_addr":"172.21.15.92:6829/2924722109","heartbeat_front_addr":"172.21.15.92:6825/2924722109","state":["exists","up"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6809","nonce":1273528383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6811","nonce":1273528383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6815","nonce":1273528383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6813","nonce":1273528383}]},"public_addr":"172.21.15.92:6809/1273528383","cluster_addr":"172.21.15.92:6811/1273528383","heartbeat_back_addr":"172.21.15.92:6815/1273528383","heartbeat_front_addr":"172.21.15.92:6813/1273528383","state":["exists","up"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.405502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.334094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.367807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.328752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.255936+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.298943+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.352632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:49:43.900 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T23:49:43.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T23:49:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:44.183 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:49:43.250751+0000","last_up_change":"2024-04-23T23:49:35.179323+0000","last_in_change":"2024-04-23T23:49:26.183177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T23:49:35.387984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T23:49:41.909447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6820","nonce":3364976130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6824","nonce":3364976130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6831","nonce":3364976130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6828","nonce":3364976130}]},"public_addr":"172.21.15.92:6820/3364976130","cluster_addr":"172.21.15.92:6824/3364976130","heartbeat_back_addr":"172.21.15.92:6831/3364976130","heartbeat_front_addr":"172.21.15.92:6828/3364976130","state":["exists","up"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6817","nonce":2924722109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6821","nonce":2924722109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6829","nonce":2924722109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6823","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6825","nonce":2924722109}]},"public_addr":"172.21.15.92:6817/2924722109","cluster_addr":"172.21.15.92:6821/2924722109","heartbeat_back_addr":"172.21.15.92:6829/2924722109","heartbeat_front_addr":"172.21.15.92:6825/2924722109","state":["exists","up"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6809","nonce":1273528383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6811","nonce":1273528383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6815","nonce":1273528383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1273528383},{"type":"v1","addr":"172.21.15.92:6813","nonce":1273528383}]},"public_addr":"172.21.15.92:6809/1273528383","cluster_addr":"172.21.15.92:6811/1273528383","heartbeat_back_addr":"172.21.15.92:6815/1273528383","heartbeat_front_addr":"172.21.15.92:6813/1273528383","state":["exists","up"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.405502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.334094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.367807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.328752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.255936+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.298943+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.352632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:49:44.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T23:49:44.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T23:49:44.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T23:49:44.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T23:49:44.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-23T23:49:44.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-23T23:49:44.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-23T23:49:44.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-23T23:49:44.436 INFO:teuthology.orchestra.run.smithi092.stdout:60129542149 2024-04-23T23:49:44.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T23:49:44.437 INFO:teuthology.orchestra.run.smithi092.stdout:64424509445 2024-04-23T23:49:44.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T23:49:44.439 INFO:teuthology.orchestra.run.smithi092.stdout:60129542149 2024-04-23T23:49:44.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T23:49:44.440 INFO:teuthology.orchestra.run.smithi092.stdout:60129542150 2024-04-23T23:49:44.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T23:49:44.441 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-23T23:49:44.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T23:49:44.442 INFO:teuthology.orchestra.run.smithi092.stdout:64424509446 2024-04-23T23:49:44.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T23:49:44.446 INFO:teuthology.orchestra.run.smithi092.stdout:60129542148 2024-04-23T23:49:44.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T23:49:44.457 INFO:teuthology.orchestra.run.smithi092.stdout:64424509446 2024-04-23T23:49:44.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T23:49:45.032 INFO:teuthology.orchestra.run.smithi092.stdout:60129542147 2024-04-23T23:49:45.036 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-04-23T23:49:45.046 INFO:teuthology.orchestra.run.smithi092.stdout:60129542148 2024-04-23T23:49:45.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.7 2024-04-23T23:49:45.056 INFO:teuthology.orchestra.run.smithi092.stdout:60129542147 2024-04-23T23:49:45.057 INFO:teuthology.orchestra.run.smithi092.stdout:60129542146 2024-04-23T23:49:45.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.0 2024-04-23T23:49:45.062 INFO:teuthology.orchestra.run.smithi092.stdout:64424509445 2024-04-23T23:49:45.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.5 2024-04-23T23:49:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:55834574850 2024-04-23T23:49:45.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.6 2024-04-23T23:49:45.075 INFO:teuthology.orchestra.run.smithi092.stdout:64424509445 2024-04-23T23:49:45.078 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.4 2024-04-23T23:49:45.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-04-23T23:49:45.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.2 2024-04-23T23:49:45.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-04-23T23:49:46.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T23:49:46.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T23:49:46.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T23:49:46.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T23:49:46.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T23:49:46.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T23:49:46.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T23:49:46.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T23:49:46.619 INFO:teuthology.orchestra.run.smithi092.stdout:60129542149 2024-04-23T23:49:46.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-04-23T23:49:46.639 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.672 INFO:teuthology.orchestra.run.smithi092.stdout:64424509446 2024-04-23T23:49:46.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-04-23T23:49:46.686 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.709 INFO:teuthology.orchestra.run.smithi092.stdout:60129542150 2024-04-23T23:49:46.710 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-23T23:49:46.712 INFO:teuthology.orchestra.run.smithi092.stdout:64424509446 2024-04-23T23:49:46.715 INFO:teuthology.orchestra.run.smithi092.stdout:60129542148 2024-04-23T23:49:46.721 INFO:teuthology.orchestra.run.smithi092.stdout:64424509445 2024-04-23T23:49:46.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-04-23T23:49:46.731 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-04-23T23:49:46.733 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-04-23T23:49:46.734 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.735 INFO:teuthology.orchestra.run.smithi092.stdout:60129542149 2024-04-23T23:49:46.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.6 2024-04-23T23:49:46.737 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.0 2024-04-23T23:49:46.740 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-04-23T23:49:46.749 DEBUG:teuthology.parallel:result is None 2024-04-23T23:49:46.749 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T23:49:46.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:49:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:47.067 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:49:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-23T23:49:45.358234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218876,"kb_used_data":2696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755844,"statfs":{"total":773094113280,"available":772869984256,"internally_reserved":0,"allocated":2760704,"data_stored":1697392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000896"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.261527+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:49:43.261527+0000","last_peered":"2024-04-23T23:49:43.261527+0000","last_clean":"2024-04-23T23:49:43.261527+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:49:43.261527+0000","last_undegraded":"2024-04-23T23:49:43.261527+0000","last_fullsized":"2024-04-23T23:49:43.261527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:25:55.772589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263265+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:49:44.263265+0000","last_peered":"2024-04-23T23:49:44.263265+0000","last_clean":"2024-04-23T23:49:44.263265+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:49:44.263265+0000","last_undegraded":"2024-04-23T23:49:44.263265+0000","last_fullsized":"2024-04-23T23:49:44.263265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266261+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:49:43.266261+0000","last_peered":"2024-04-23T23:49:43.266261+0000","last_clean":"2024-04-23T23:49:43.266261+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:49:43.266261+0000","last_undegraded":"2024-04-23T23:49:43.266261+0000","last_fullsized":"2024-04-23T23:49:43.266261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:50:42.432476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263350+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:49:44.263350+0000","last_peered":"2024-04-23T23:49:44.263350+0000","last_clean":"2024-04-23T23:49:44.263350+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:49:44.263350+0000","last_undegraded":"2024-04-23T23:49:44.263350+0000","last_fullsized":"2024-04-23T23:49:44.263350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265330+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:49:43.265330+0000","last_peered":"2024-04-23T23:49:43.265330+0000","last_clean":"2024-04-23T23:49:43.265330+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:49:43.265330+0000","last_undegraded":"2024-04-23T23:49:43.265330+0000","last_fullsized":"2024-04-23T23:49:43.265330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:02:56.770400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.282470+0000","last_change":"2024-04-23T23:49:43.280465+0000","last_active":"2024-04-23T23:49:43.282470+0000","last_peered":"2024-04-23T23:49:43.282470+0000","last_clean":"2024-04-23T23:49:43.282470+0000","last_became_active":"2024-04-23T23:49:43.280144+0000","last_became_peered":"2024-04-23T23:49:43.280144+0000","last_unstale":"2024-04-23T23:49:43.282470+0000","last_undegraded":"2024-04-23T23:49:43.282470+0000","last_fullsized":"2024-04-23T23:49:43.282470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:45:09.679230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266288+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:49:43.266288+0000","last_peered":"2024-04-23T23:49:43.266288+0000","last_clean":"2024-04-23T23:49:43.266288+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:49:43.266288+0000","last_undegraded":"2024-04-23T23:49:43.266288+0000","last_fullsized":"2024-04-23T23:49:43.266288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:58.237999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265346+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:49:43.265346+0000","last_peered":"2024-04-23T23:49:43.265346+0000","last_clean":"2024-04-23T23:49:43.265346+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:49:43.265346+0000","last_undegraded":"2024-04-23T23:49:43.265346+0000","last_fullsized":"2024-04-23T23:49:43.265346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:00.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.256802+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:49:43.256802+0000","last_peered":"2024-04-23T23:49:43.256802+0000","last_clean":"2024-04-23T23:49:43.256802+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:49:43.256802+0000","last_undegraded":"2024-04-23T23:49:43.256802+0000","last_fullsized":"2024-04-23T23:49:43.256802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":654950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:49:47.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:49:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:47.363 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:49:47.374 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-23T23:49:47.358527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218924,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755796,"statfs":{"total":773094113280,"available":772869935104,"internally_reserved":0,"allocated":2797568,"data_stored":1714636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001189"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.261527+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:49:43.261527+0000","last_peered":"2024-04-23T23:49:43.261527+0000","last_clean":"2024-04-23T23:49:43.261527+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:49:43.261527+0000","last_undegraded":"2024-04-23T23:49:43.261527+0000","last_fullsized":"2024-04-23T23:49:43.261527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:25:55.772589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263265+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:49:44.263265+0000","last_peered":"2024-04-23T23:49:44.263265+0000","last_clean":"2024-04-23T23:49:44.263265+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:49:44.263265+0000","last_undegraded":"2024-04-23T23:49:44.263265+0000","last_fullsized":"2024-04-23T23:49:44.263265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266261+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:49:43.266261+0000","last_peered":"2024-04-23T23:49:43.266261+0000","last_clean":"2024-04-23T23:49:43.266261+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:49:43.266261+0000","last_undegraded":"2024-04-23T23:49:43.266261+0000","last_fullsized":"2024-04-23T23:49:43.266261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:50:42.432476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263350+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:49:44.263350+0000","last_peered":"2024-04-23T23:49:44.263350+0000","last_clean":"2024-04-23T23:49:44.263350+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:49:44.263350+0000","last_undegraded":"2024-04-23T23:49:44.263350+0000","last_fullsized":"2024-04-23T23:49:44.263350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265330+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:49:43.265330+0000","last_peered":"2024-04-23T23:49:43.265330+0000","last_clean":"2024-04-23T23:49:43.265330+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:49:43.265330+0000","last_undegraded":"2024-04-23T23:49:43.265330+0000","last_fullsized":"2024-04-23T23:49:43.265330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:02:56.770400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.282470+0000","last_change":"2024-04-23T23:49:43.280465+0000","last_active":"2024-04-23T23:49:43.282470+0000","last_peered":"2024-04-23T23:49:43.282470+0000","last_clean":"2024-04-23T23:49:43.282470+0000","last_became_active":"2024-04-23T23:49:43.280144+0000","last_became_peered":"2024-04-23T23:49:43.280144+0000","last_unstale":"2024-04-23T23:49:43.282470+0000","last_undegraded":"2024-04-23T23:49:43.282470+0000","last_fullsized":"2024-04-23T23:49:43.282470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:45:09.679230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266288+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:49:43.266288+0000","last_peered":"2024-04-23T23:49:43.266288+0000","last_clean":"2024-04-23T23:49:43.266288+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:49:43.266288+0000","last_undegraded":"2024-04-23T23:49:43.266288+0000","last_fullsized":"2024-04-23T23:49:43.266288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:58.237999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265346+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:49:43.265346+0000","last_peered":"2024-04-23T23:49:43.265346+0000","last_clean":"2024-04-23T23:49:43.265346+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:49:43.265346+0000","last_undegraded":"2024-04-23T23:49:43.265346+0000","last_fullsized":"2024-04-23T23:49:43.265346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:00.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.256802+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:49:43.256802+0000","last_peered":"2024-04-23T23:49:43.256802+0000","last_clean":"2024-04-23T23:49:43.256802+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:49:43.256802+0000","last_undegraded":"2024-04-23T23:49:43.256802+0000","last_fullsized":"2024-04-23T23:49:43.256802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":654950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:49:47.375 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T23:49:47.375 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T23:49:47.375 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-23T23:49:47.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T23:49:47.688 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:47.689 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T23:49:47.701 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-23T23:49:47.701 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-23T23:49:47.712 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-23T23:49:47.712 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b98aab6c00f483888c049d0d8e7bc507e115dd86'}} 2024-04-23T23:49:47.712 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-23T23:49:47.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-23T23:49:48.011 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-23T23:49:48.011 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T23:49:48.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:49:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:48.292 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:49:48.304 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-23T23:49:47.358527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218924,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755796,"statfs":{"total":773094113280,"available":772869935104,"internally_reserved":0,"allocated":2797568,"data_stored":1714636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001189"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.261527+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:49:43.261527+0000","last_peered":"2024-04-23T23:49:43.261527+0000","last_clean":"2024-04-23T23:49:43.261527+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:49:43.261527+0000","last_undegraded":"2024-04-23T23:49:43.261527+0000","last_fullsized":"2024-04-23T23:49:43.261527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:25:55.772589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263265+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:49:44.263265+0000","last_peered":"2024-04-23T23:49:44.263265+0000","last_clean":"2024-04-23T23:49:44.263265+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:49:44.263265+0000","last_undegraded":"2024-04-23T23:49:44.263265+0000","last_fullsized":"2024-04-23T23:49:44.263265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266261+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:49:43.266261+0000","last_peered":"2024-04-23T23:49:43.266261+0000","last_clean":"2024-04-23T23:49:43.266261+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:49:43.266261+0000","last_undegraded":"2024-04-23T23:49:43.266261+0000","last_fullsized":"2024-04-23T23:49:43.266261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:50:42.432476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263350+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:49:44.263350+0000","last_peered":"2024-04-23T23:49:44.263350+0000","last_clean":"2024-04-23T23:49:44.263350+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:49:44.263350+0000","last_undegraded":"2024-04-23T23:49:44.263350+0000","last_fullsized":"2024-04-23T23:49:44.263350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265330+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:49:43.265330+0000","last_peered":"2024-04-23T23:49:43.265330+0000","last_clean":"2024-04-23T23:49:43.265330+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:49:43.265330+0000","last_undegraded":"2024-04-23T23:49:43.265330+0000","last_fullsized":"2024-04-23T23:49:43.265330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:02:56.770400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.282470+0000","last_change":"2024-04-23T23:49:43.280465+0000","last_active":"2024-04-23T23:49:43.282470+0000","last_peered":"2024-04-23T23:49:43.282470+0000","last_clean":"2024-04-23T23:49:43.282470+0000","last_became_active":"2024-04-23T23:49:43.280144+0000","last_became_peered":"2024-04-23T23:49:43.280144+0000","last_unstale":"2024-04-23T23:49:43.282470+0000","last_undegraded":"2024-04-23T23:49:43.282470+0000","last_fullsized":"2024-04-23T23:49:43.282470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:45:09.679230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266288+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:49:43.266288+0000","last_peered":"2024-04-23T23:49:43.266288+0000","last_clean":"2024-04-23T23:49:43.266288+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:49:43.266288+0000","last_undegraded":"2024-04-23T23:49:43.266288+0000","last_fullsized":"2024-04-23T23:49:43.266288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:58.237999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265346+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:49:43.265346+0000","last_peered":"2024-04-23T23:49:43.265346+0000","last_clean":"2024-04-23T23:49:43.265346+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:49:43.265346+0000","last_undegraded":"2024-04-23T23:49:43.265346+0000","last_fullsized":"2024-04-23T23:49:43.265346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:00.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.256802+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:49:43.256802+0000","last_peered":"2024-04-23T23:49:43.256802+0000","last_clean":"2024-04-23T23:49:43.256802+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:49:43.256802+0000","last_undegraded":"2024-04-23T23:49:43.256802+0000","last_fullsized":"2024-04-23T23:49:43.256802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":654950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:49:48.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:49:48.585 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:48.586 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:49:48.597 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-23T23:49:47.358527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218924,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755796,"statfs":{"total":773094113280,"available":772869935104,"internally_reserved":0,"allocated":2797568,"data_stored":1714636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001189"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.261527+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:49:43.261527+0000","last_peered":"2024-04-23T23:49:43.261527+0000","last_clean":"2024-04-23T23:49:43.261527+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:49:43.261527+0000","last_undegraded":"2024-04-23T23:49:43.261527+0000","last_fullsized":"2024-04-23T23:49:43.261527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:25:55.772589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263265+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:49:44.263265+0000","last_peered":"2024-04-23T23:49:44.263265+0000","last_clean":"2024-04-23T23:49:44.263265+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:49:44.263265+0000","last_undegraded":"2024-04-23T23:49:44.263265+0000","last_fullsized":"2024-04-23T23:49:44.263265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266261+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:49:43.266261+0000","last_peered":"2024-04-23T23:49:43.266261+0000","last_clean":"2024-04-23T23:49:43.266261+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:49:43.266261+0000","last_undegraded":"2024-04-23T23:49:43.266261+0000","last_fullsized":"2024-04-23T23:49:43.266261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:50:42.432476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T23:49:44.263350+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:49:44.263350+0000","last_peered":"2024-04-23T23:49:44.263350+0000","last_clean":"2024-04-23T23:49:44.263350+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:49:44.263350+0000","last_undegraded":"2024-04-23T23:49:44.263350+0000","last_fullsized":"2024-04-23T23:49:44.263350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265330+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:49:43.265330+0000","last_peered":"2024-04-23T23:49:43.265330+0000","last_clean":"2024-04-23T23:49:43.265330+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:49:43.265330+0000","last_undegraded":"2024-04-23T23:49:43.265330+0000","last_fullsized":"2024-04-23T23:49:43.265330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:02:56.770400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.282470+0000","last_change":"2024-04-23T23:49:43.280465+0000","last_active":"2024-04-23T23:49:43.282470+0000","last_peered":"2024-04-23T23:49:43.282470+0000","last_clean":"2024-04-23T23:49:43.282470+0000","last_became_active":"2024-04-23T23:49:43.280144+0000","last_became_peered":"2024-04-23T23:49:43.280144+0000","last_unstale":"2024-04-23T23:49:43.282470+0000","last_undegraded":"2024-04-23T23:49:43.282470+0000","last_fullsized":"2024-04-23T23:49:43.282470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:45:09.679230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.266288+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:49:43.266288+0000","last_peered":"2024-04-23T23:49:43.266288+0000","last_clean":"2024-04-23T23:49:43.266288+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:49:43.266288+0000","last_undegraded":"2024-04-23T23:49:43.266288+0000","last_fullsized":"2024-04-23T23:49:43.266288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:58.237999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.265346+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:49:43.265346+0000","last_peered":"2024-04-23T23:49:43.265346+0000","last_clean":"2024-04-23T23:49:43.265346+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:49:43.265346+0000","last_undegraded":"2024-04-23T23:49:43.265346+0000","last_fullsized":"2024-04-23T23:49:43.265346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:00.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T23:49:43.256802+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:49:43.256802+0000","last_peered":"2024-04-23T23:49:43.256802+0000","last_clean":"2024-04-23T23:49:43.256802+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:49:43.256802+0000","last_undegraded":"2024-04-23T23:49:43.256802+0000","last_fullsized":"2024-04-23T23:49:43.256802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":654950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:49:48.598 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T23:49:48.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-23T23:49:48.878 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 21 2024-04-23T23:49:48.878 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 6a51c05a-822c-4999-80c7-e8408cd2a202 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-04-23T23:49:23.155452+0000 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-04-23T23:49:44.251971+0000 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 5 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2024-04-23T23:49:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client jewel 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release squid 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 application rbd read_balance_score 1.99 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 8 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/3364976130,v1:172.21.15.92:6820/3364976130] [v2:172.21.15.92:6822/3364976130,v1:172.21.15.92:6824/3364976130] exists,up 6a84eb3e-7d8a-4436-956d-f7da68d06a21 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/2924722109,v1:172.21.15.92:6817/2924722109] [v2:172.21.15.92:6819/2924722109,v1:172.21.15.92:6821/2924722109] exists,up be35c480-9a84-4846-9f0b-043c7c2c9f4a 2024-04-23T23:49:48.880 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3952319845,v1:172.21.15.92:6801/3952319845] [v2:172.21.15.92:6802/3952319845,v1:172.21.15.92:6803/3952319845] exists,up 64e8bbbb-690a-4267-a030-30ec66f30bfa 2024-04-23T23:49:48.881 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1273528383,v1:172.21.15.92:6809/1273528383] [v2:172.21.15.92:6810/1273528383,v1:172.21.15.92:6811/1273528383] exists,up 692beda9-1302-4d96-81ab-62c40a12e3f1 2024-04-23T23:49:48.881 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/1281396523,v1:172.21.15.119:6825/1281396523] [v2:172.21.15.119:6826/1281396523,v1:172.21.15.119:6827/1281396523] exists,up 705f6b1f-c35d-4ed8-a2c9-70b9324691e3 2024-04-23T23:49:48.881 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/612886539,v1:172.21.15.119:6801/612886539] [v2:172.21.15.119:6802/612886539,v1:172.21.15.119:6803/612886539] exists,up eb37446d-b5c9-4200-a1e9-38c1f6268401 2024-04-23T23:49:48.881 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/2243867625,v1:172.21.15.119:6817/2243867625] [v2:172.21.15.119:6818/2243867625,v1:172.21.15.119:6819/2243867625] exists,up c38a4da5-1017-4bb9-86ab-999ea57999c9 2024-04-23T23:49:48.881 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/3209176445,v1:172.21.15.119:6809/3209176445] [v2:172.21.15.119:6810/3209176445,v1:172.21.15.119:6811/3209176445] exists,up a921d465-920a-4ad0-839f-d97cd5e6ce62 2024-04-23T23:49:48.891 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/3364976130,v1:172.21.15.92:6820/3364976130] [v2:172.21.15.92:6822/3364976130,v1:172.21.15.92:6824/3364976130] exists,up 6a84eb3e-7d8a-4436-956d-f7da68d06a21', 'osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/2924722109,v1:172.21.15.92:6817/2924722109] [v2:172.21.15.92:6819/2924722109,v1:172.21.15.92:6821/2924722109] exists,up be35c480-9a84-4846-9f0b-043c7c2c9f4a', 'osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3952319845,v1:172.21.15.92:6801/3952319845] [v2:172.21.15.92:6802/3952319845,v1:172.21.15.92:6803/3952319845] exists,up 64e8bbbb-690a-4267-a030-30ec66f30bfa', 'osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1273528383,v1:172.21.15.92:6809/1273528383] [v2:172.21.15.92:6810/1273528383,v1:172.21.15.92:6811/1273528383] exists,up 692beda9-1302-4d96-81ab-62c40a12e3f1', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/1281396523,v1:172.21.15.119:6825/1281396523] [v2:172.21.15.119:6826/1281396523,v1:172.21.15.119:6827/1281396523] exists,up 705f6b1f-c35d-4ed8-a2c9-70b9324691e3', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/612886539,v1:172.21.15.119:6801/612886539] [v2:172.21.15.119:6802/612886539,v1:172.21.15.119:6803/612886539] exists,up eb37446d-b5c9-4200-a1e9-38c1f6268401', 'osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/2243867625,v1:172.21.15.119:6817/2243867625] [v2:172.21.15.119:6818/2243867625,v1:172.21.15.119:6819/2243867625] exists,up c38a4da5-1017-4bb9-86ab-999ea57999c9', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/3209176445,v1:172.21.15.119:6809/3209176445] [v2:172.21.15.119:6810/3209176445,v1:172.21.15.119:6811/3209176445] exists,up a921d465-920a-4ad0-839f-d97cd5e6ce62'] 2024-04-23T23:49:48.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-23T23:49:48.988 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:48.988 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mon.a", 2024-04-23T23:49:48.988 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster": "ceph", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket_mode": "", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_client_required": "cephx, none", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_cluster_required": "cephx", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_debug": "true", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_required": "cephx", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_supported": "cephx", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio": "true", 2024-04-23T23:49:48.989 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_poll_ms": "250", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_reap_max": "16", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_async_discard": "false", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_block_size": "4096", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio": "false", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_inflight_ios": "false", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_enable_discard": "false", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry": "3", 2024-04-23T23:49:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash": "0", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring": "false", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring_hipri": "false", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_type": "", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_alloc_size": "1048576", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_allocator": "hybrid", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_buffered_io": "true", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_for_zeros": "false", 2024-04-23T23:49:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_compact_log_sync": "false", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_log_runway": "4194304", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_prefetch": "1048576", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_flush_size": "524288", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_log_runway": "1048576", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery": "false", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-23T23:49:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_sync_write": "false", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocation_from_file": "true", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocator": "hybrid", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_create": "true", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_create": "false", 2024-04-23T23:49:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_path": "", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_size": "0", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_path": "", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_preallocate_file": "false", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_size": "107374182400", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_create": "false", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_path": "", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_size": "100663296", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs": "true", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-23T23:49:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune": "true", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size": "0", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_type": "2q", 2024-04-23T23:49:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_clone_cow": "true", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_mode": "none", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_csum_type": "crc32c", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_freelist": "false", 2024-04-23T23:49:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_small_allocations": "0", 2024-04-23T23:49:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_read": "true", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_write": "false", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-23T23:49:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fail_eio": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-23T23:49:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount": "false", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-23T23:49:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_ignore_data_csum": "false", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_op_age": "5.000000", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_alloc_size": "0", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size": "0", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-23T23:49:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_deferred_txc": "32", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size": "0", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_nid_prealloc": "1024", 2024-04-23T23:49:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_retry_disk_reads": "3", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cf": "true", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-23T23:49:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_mem": "512", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-23T23:49:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_tracing": "false", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-23T23:49:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_zero_block_detection": "false", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-23T23:49:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-23T23:49:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_signatures": "false", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_version": "2", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_signatures": "false", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_version": "2", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_signatures": "false", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_version": "2", 2024-04-23T23:49:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_sign_messages": "true", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "chdir": "", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_acl_type": "", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_asio_thread_count": "2", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_mid": "0.750000", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_size": "16384", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_caps_release_delay": "5", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_check_pool_perm": "true", 2024-04-23T23:49:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_force_sync_read": "false", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_getattr_caps": "false", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_inject_tick_delay": "0", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_remount": "false", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_dirsize_rbytes": "true", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_force_lazyio": "false", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_fs": "", 2024-04-23T23:49:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_release_failure": "false", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_inline_size": "4096", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mds_namespace": "", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_metadata": "", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_gid": "-1", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_timeout": "300", 2024-04-23T23:49:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_uid": "-1", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mountpoint": "/", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_notify_timeout": "10", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc": "true", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty": "104857600", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_objects": "1000", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_size": "209715200", 2024-04-23T23:49:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_target_dirty": "8388608", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_permissions": "true", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota": "true", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota_df": "true", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_bytes": "0", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_periods": "4", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_min": "131072", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_reconnect_stale": "false", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_shutdown_timeout": "30", 2024-04-23T23:49:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_snapdir": ".snap", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_tick_interval": "1", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_trace": "", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_try_dentry_invalidate": "false", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_faked_inos": "false", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_random_mds": "false", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog": "false", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_port": "12201", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_monitors": "default=true", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog": "false", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_level": "info", 2024-04-23T23:49:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "-", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network": "", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network_interface": "", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_isal": "false", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_level": "5", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_winsize": "-15", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zstd_level": "1", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_alien_op_num_threads": "6", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_seastar_cpu_cores": "", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location": "", 2024-04-23T23:49:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook": "", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook_timeout": "10", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "daemonize": "false", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "daos_pool": "tank", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_alienstore": "0/5", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_allow_any_pool_priority": "false", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok": "1/5", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok_assert_abort": "false", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asserts_on_shutdown": "true", 2024-04-23T23:49:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_auth": "20/20", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bdev": "1/3", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluefs": "1/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluestore": "1/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_buffer": "0/1", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ceph_exporter": "1/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephfs_mirror": "0/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephsqlite": "0/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_client": "0/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_compressor": "1/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_context": "0/1", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crimson_interrupt": "0/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crush": "1/1", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crypto": "1/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cyanstore": "0/5", 2024-04-23T23:49:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_deliberately_leak_memory": "false", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_disable_randomized_ping": "false", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_dpdk": "1/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_eventtrace": "1/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filer": "0/1", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filestore": "1/3", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_finisher": "1/1", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_fuse": "1/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeat_testing_span": "0", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeatmap": "1/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_javaclient": "1/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journal": "1/3", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journaler": "0/5", 2024-04-23T23:49:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_kstore": "1/5", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_lockdep": "0/1", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mclock": "1/5", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds": "1/5", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_balancer": "1/5", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_locker": "1/5", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log": "1/5", 2024-04-23T23:49:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log_expire": "1/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_migrator": "1/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_quiesce": "3/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_memstore": "1/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgr": "2/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgrc": "1/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mon": "20/20", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_monc": "0/10", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ms": "1/1", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_none": "0/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objclass": "0/5", 2024-04-23T23:49:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objectcacher": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objecter": "0/1", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_optracker": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_osd": "1/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_paxos": "20/20", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_perfcounter": "1/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_prioritycache": "1/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rados": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_mirror": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_pwl": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_replay": "0/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_refs": "0/0", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_reserver": "1/1", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw": "1/5", 2024-04-23T23:49:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_access": "1/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_datacache": "1/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_dbstore": "1/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_flight": "1/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_sync": "1/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rocksdb": "4/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore": "0/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_backref": "0/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_cache": "0/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_cleaner": "0/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_device": "0/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_epm": "0/5", 2024-04-23T23:49:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_journal": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_lba": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_odata": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_omap": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_onode": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_t": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_tm": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_striper": "0/1", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_test": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_throttle": "1/1", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_timer": "0/1", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_tp": "0/5", 2024-04-23T23:49:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_trace": "1/5", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "device_failure_prediction_mode": "none", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_graylog": "false", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_journald": "false", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_stderr": "true", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_syslog": "false", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event_tracing": "false", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_addr": "0.0.0.0", 2024-04-23T23:49:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_http_port": "9926", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_prio_limit": "5", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_sort_metrics": "true", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_stats_period": "5", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "fake_statfs_for_testing": "0", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "fatal_signal_handlers": "true", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_purge_ops": "10", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_truncate_ops": "128", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_apply_finisher_threads": "1", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_blackhole": "false", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_clone_range": "true", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_snap": "true", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_caller_concurrency": "10", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_collect_device_partition_information": "true", 2024-04-23T23:49:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_commit_timeout": "600.000000", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_inject_read_err": "false", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_omap_check": "false", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_verify_split": "false", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_dump_file": "", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fadvise": "true", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fail_eio": "true", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_shards": "16", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_size": "128", 2024-04-23T23:49:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap": "false", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap_threshold": "4096", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_inject_stall": "0", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_parallel": "false", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_trailing": "false", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_writeahead": "false", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_kill_at": "0", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs": "0", 2024-04-23T23:49:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size": "0", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-23T23:49:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_merge_threshold": "-10", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_odsync_write": "false", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend": "rocksdb", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend_path": "", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_timeout": "60", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_threads": "2", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_punch_hole": "false", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-23T23:49:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_ops": "50", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_seek_data_hole": "false", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc": "false", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-23T23:49:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_splice": "false", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_multiple": "2", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_rand_factor": "20", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_update_to": "1000", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_enable": "true", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-23T23:49:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_xfs_extsize": "false", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_zfs_snap": "false", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fio_dir": "/tmp/fio", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "6a51c05a-822c-4999-80c7-e8408cd2a202", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_allow_other": "true", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_atomic_o_trunc": "true", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_big_writes": "true", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_debug": "false", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_default_permissions": "false", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_disable_pagecache": "false", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_max_write": "0", 2024-04-23T23:49:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_multithreaded": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_require_active_mds": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_set_user_groups": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_move": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_read": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_write": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_use_invalidate_cb": "true", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_target_name": "ceph", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_file": "", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_inject_failure": "0", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_interval": "5", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "host": "smithi092", 2024-04-23T23:49:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-23T23:49:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "inject_early_sigterm": "false", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "jaeger_agent_port": "6799", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "jaeger_tracing_enable": "false", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_aio": "true", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_align_min_size": "65536", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_align": "true", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_size": "4096", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_dio": "true", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_discard": "false", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_force_aio": "false", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_ignore_corruption": "false", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_bytes": "10485760", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_entries": "100", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_replay_from": "0", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-23T23:49:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_write_header_frequency": "0", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_zero_on_create": "false", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prefetch_periods": "10", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prezero_periods": "5", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_write_head_interval": "15", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "keyfile": "", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_backend": "rocksdb", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_default_stripe_size": "65536", 2024-04-23T23:49:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount": "false", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_bytes": "67108864", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_ops": "512", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_nid_prealloc": "1024", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_onode_map_size": "1024", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_submit_transaction": "false", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_transaction": "false", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "librados_thread_count": "2", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep": "false", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep_force_backtrace": "false", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "log_coarse_timestamps": "true", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "log_flush_on_exit": "false", 2024-04-23T23:49:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_host": "127.0.0.1", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_port": "12201", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_new": "1000", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_recent": "10000", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stderr_prefix": "", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stop_at_utilization": "0.970000", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": "true", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_graylog": "false", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_journald": "false", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_stderr": "false", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_syslog": "false", 2024-04-23T23:49:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "max_rotating_auth_attempts": "10", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_action_on_write_error": "1", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_alternate_name_max": "8192", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_asio_thread_count": "2", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_export_pin": "true", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_dirs": "true", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_interval": "5", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_interval": "10", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max": "-1", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max_until": "-1", 2024-04-23T23:49:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_merge_size": "50", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_midchunk": "0.300000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_start": "0.200000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_minchunk": "0.001000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_mode": "0", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_max": "1.200000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_min": "0.800000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_overload_epochs": "2", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-23T23:49:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_bits": "3", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_size": "10000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_target_decay": "10.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_grace": "15.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_interval": "4.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_mid": "0.700000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_delay": "0", 2024-04-23T23:49:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_release_free_interval": "10", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_reservation": "0.050000", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_interval": "1", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_threshold": "262144", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_prealloc_inos": "1000", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_connect_bootstrapping": "false", 2024-04-23T23:49:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_damage_table_max_entries": "10000", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_auth_pins": "false", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_frag": "false", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_scatterstat": "false", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_subtrees": "false", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_decay_halflife": "5.000000", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_default_dir_hash": "2", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_defer_session_stale": "true", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_deny_all_reconnect": "false", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_keys_per_op": "16384", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_commit_size": "10", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_entries": "0", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_prefetch": "true", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-23T23:49:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_on_map": "false", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_early_reply": "true", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enable_op_tracker": "true", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enforce_unique_name": "true", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random": "true", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-23T23:49:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_summarize_threshold": "10", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_health_dummy": "false", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_migrator_session_race": "false", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_join_fs": "", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_journal_format": "1", 2024-04-23T23:49:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_create_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_export_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_import_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_expire_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_replay_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_link_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_mdstable_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_openc_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_rename_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_shutdown_at": "0", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_event_large_threshold": "524288", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_events_per_segment": "1024", 2024-04-23T23:49:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_events": "-1", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_segments": "128", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_pause": "false", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_segment_size": "0", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_trim_threshold": "128", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_warn_factor": "2.000000", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_caps_per_client": "1048576", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_flushes": "100000", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_requests": "100000", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_export_size": "20971520", 2024-04-23T23:49:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_file_recover": "32", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_files": "64", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops": "8192", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_snaps_per_dir": "100", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_metrics_update_interval": "2", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_per_client": "100", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_working_set": "10000", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_numa_node": "-1", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_complaint_time": "30.000000", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_duration": "600", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_size": "20", 2024-04-23T23:49:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_slow_op_size": "20", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_log_threshold": "5", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_grace": "15", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_interval": "5", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_caps": "30000", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_threshold": "262144", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_interval": "1.000000", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-23T23:49:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_gid": "0", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_uid": "0", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_shutdown_check": "0", 2024-04-23T23:49:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_skip_ino": "0", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_max_uid": "4294967294", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_min_uid": "0", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_rstat": "false", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_standby_replay_damaged": "false", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_symlink_recovery": "true", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_exports": "0", 2024-04-23T23:49:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_fragments": "0", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_tick_interval": "5.000000", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_valgrind_exit": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_backtrace": "1", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_scatter": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_sessions": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool_debug": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_device_bytes": "1073741824", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_set": "false", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_size": "65536", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_bytes": "134217728", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_messages": "512", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-23T23:49:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_disabled_modules": "", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_max_pg_num_change": "128", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_bytes": "134217728", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_messages": "128", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_bytes": "134217728", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_messages": "128", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_bytes": "536870912", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_messages": "8192", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_pool": "true", 2024-04-23T23:49:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_standby_modules": "true", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_period": "5", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_threshold": "5", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_tick_period": "2", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_delete": "true", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_size_one": "true", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_auth_validate_all_caps": "true", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_bytes": "104857600", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_directed_command_retry": "5", 2024-04-23T23:49:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_parallel": "3", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_log_interval": "1.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_interval": "10.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_target_rank": "-1", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_file_level": "debug", 2024-04-23T23:49:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_file": "true", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_journald": "false", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-23T23:49:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-23T23:49:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_bootstrap": "false", 2024-04-23T23:49:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_start": "false", 2024-04-23T23:49:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_trim": "true", 2024-04-23T23:49:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-23T23:49:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-23T23:49:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-23T23:49:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cpu_threads": "4", 2024-04-23T23:49:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_crush_min_required_version": "hammer", 2024-04-23T23:49:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_daemon_bytes": "419430400", 2024-04-23T23:49:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-23T23:49:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_crit": "5", 2024-04-23T23:49:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_warn": "5", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_size_warn": "16106127360", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_json": "false", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_transactions": "false", 2024-04-23T23:49:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_extra_checks": "false", 2024-04-23T23:49:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-23T23:49:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-23T23:49:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_reef": "false", 2024-04-23T23:49:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_squid": "false", 2024-04-23T23:49:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-23T23:49:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-23T23:49:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-23T23:49:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_down_mkfs_grace": "120", 2024-04-23T23:49:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_down_uptime_grace": "60", 2024-04-23T23:49:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_default_strategy": "1", 2024-04-23T23:49:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_timeout": "5.000000", 2024-04-23T23:49:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-23T23:49:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_divisor": "2", 2024-04-23T23:49:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-23T23:49:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_enable_op_tracker": "true", 2024-04-23T23:49:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_fake_pool_delete": "false", 2024-04-23T23:49:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_force_quorum_join": "false", 2024-04-23T23:49:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_globalid_prealloc": "10000", 2024-04-23T23:49:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_detail_to_clog": "false", 2024-04-23T23:49:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_log_update_period": "5", 2024-04-23T23:49:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_max_detail": "50", 2024-04-23T23:49:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog": "true", 2024-04-23T23:49:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_interval": "600", 2024-04-23T23:49:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-23T23:49:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host": "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.119", 2024-04-23T23:49:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host_override": "", 2024-04-23T23:49:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_initial_members": "", 2024-04-23T23:49:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-23T23:49:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-23T23:49:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-23T23:49:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-23T23:49:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-23T23:49:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease": "5.000000", 2024-04-23T23:49:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-23T23:49:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-23T23:49:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_full_interval": "50", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max": "10000", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max_summary": "50", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_epochs": "500", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-23T23:49:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_osd": "10000", 2024-04-23T23:49:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pg_per_osd": "10000", 2024-04-23T23:49:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pool_pg_num": "65536", 2024-04-23T23:49:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-23T23:49:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-23T23:49:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_force_trim_to": "0", 2024-04-23T23:49:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_skip_sanity": "false", 2024-04-23T23:49:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_autotune": "true", 2024-04-23T23:49:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_target": "2147483648", 2024-04-23T23:49:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_beacon_grace": "30", 2024-04-23T23:49:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-23T23:49:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_digest_period": "5", 2024-04-23T23:49:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_inactive_grace": "60", 2024-04-23T23:49:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-23T23:49:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-23T23:49:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_min_osdmap_epochs": "500", 2024-04-23T23:49:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_complaint_time": "30", 2024-04-23T23:49:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_duration": "600", 2024-04-23T23:49:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_size": "20", 2024-04-23T23:49:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_size": "20", 2024-04-23T23:49:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-23T23:49:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_log_threshold": "5", 2024-04-23T23:49:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-23T23:49:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-23T23:49:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_in": "false", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size": "500", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-23T23:49:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-23T23:49:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-23T23:49:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_interval": "600", 2024-04-23T23:49:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-23T23:49:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-23T23:49:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_force_trim_to": "0", 2024-04-23T23:49:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-23T23:49:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-23T23:49:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-23T23:49:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-23T23:49:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-23T23:49:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-23T23:49:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-23T23:49:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-23T23:49:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_down_reporters": "2", 2024-04-23T23:49:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-23T23:49:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-23T23:49:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-23T23:49:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_report_timeout": "900", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-23T23:49:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-23T23:49:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-23T23:49:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-23T23:49:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-23T23:49:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-23T23:49:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-23T23:49:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_stuck_threshold": "60", 2024-04-23T23:49:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-23T23:49:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-23T23:49:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-23T23:49:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-23T23:49:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-23T23:49:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-23T23:49:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_probe_timeout": "2.000000", 2024-04-23T23:49:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_change": "0.050000", 2024-04-23T23:49:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_osds": "4", 2024-04-23T23:49:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-23T23:49:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-23T23:49:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-23T23:49:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-23T23:49:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-23T23:49:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_interval": "86400", 2024-04-23T23:49:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_max_keys": "100", 2024-04-23T23:49:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_timeout": "300", 2024-04-23T23:49:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_session_timeout": "300", 2024-04-23T23:49:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_smart_report_timeout": "5", 2024-04-23T23:49:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stat_smooth_intervals": "6", 2024-04-23T23:49:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-23T23:49:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_min_size": "2", 2024-04-23T23:49:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_size": "4", 2024-04-23T23:49:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-23T23:49:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-23T23:49:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_debug": "false", 2024-04-23T23:49:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-23T23:49:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-23T23:49:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_provider_kill_at": "0", 2024-04-23T23:49:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_requester_kill_at": "0", 2024-04-23T23:49:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_timeout": "60.000000", 2024-04-23T23:49:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_target_pg_per_osd": "100", 2024-04-23T23:49:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_tick_interval": "5", 2024-04-23T23:49:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_interval": "300.000000", 2024-04-23T23:49:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_use_min_delay_socket": "false", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_older_version_delay": "604800", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-23T23:49:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-23T23:49:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-23T23:49:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-23T23:49:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_misplaced": "false", 2024-04-23T23:49:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-23T23:49:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_older_version": "true", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_app": "false", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-23T23:49:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-23T23:49:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-23T23:49:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-23T23:49:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "monmap": "", 2024-04-23T23:49:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-23T23:49:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-23T23:49:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-23T23:49:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-23T23:49:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-23T23:49:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-23T23:49:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_tracing_enabled": "false", 2024-04-23T23:49:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_op_threads": "3", 2024-04-23T23:49:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_cm": "false", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_device_name": "", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_dscp": "96", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_local_gid": "", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-23T23:49:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_port_num": "1", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_sl": "3", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_support_srq": "true", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_type": "ib", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_reap_threshold": "5", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_before_connect": "false", 2024-04-23T23:49:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv4": "true", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv6": "false", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr1": "true", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr2": "true", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_max": "7568", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_min": "6800", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_count": "3", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_delay": "5", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_client": "false", 2024-04-23T23:49:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mds": "false", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mgr": "false", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mon": "false", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_osd": "false", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_client_mode": "crc secure", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_mode": "crc secure", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_type": "", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_compress_secure": "false", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_idle_timeout": "900", 2024-04-23T23:49:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_ready_timeout": "10", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_data": "true", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_header": "true", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bad_msg": "false", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bug": "true", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_old_message": "true", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_skipped_message": "false", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_coremask": "0xF", 2024-04-23T23:49:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_enable_tso": "true", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hugepages": "", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_lro": "true", 2024-04-23T23:49:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_memory_channel": "4", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_pmd": "", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_port_id": "0", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_on_send": "false", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_initial_backoff": "0.200000", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_max": "1.000000", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-23T23:49:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_type": "", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_network_congestion": "0", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_socket_failures": "5000", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_learn_addr_from_peer": "true", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_accept_failures": "4", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_backoff": "15.000000", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_client_mode": "secure crc", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-23T23:49:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_service_mode": "secure crc", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compress_min_size": "1024", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compress_mode": "none", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_min_cost": "65536", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_public_type": "", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_service_mode": "crc secure", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_listen_backlog": "512", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_nodelay": "true", 2024-04-23T23:49:49.096 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_rcvbuf": "0", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_type": "async+posix", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "no_config_file": "false", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_completion_locks_per_session": "32", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_ops": "1024", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-23T23:49:49.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_tick_interval": "5.000000", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_timeout": "10.000000", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "objectstore_blackhole": "false", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "openssl_engine_opts": "", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_delay_time": "5.000000", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_hist_halflife": "1000", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_low_ops": "2", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_ops": "4", 2024-04-23T23:49:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_slop": "0.020000", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_asio_thread_count": "2", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_async_recovery_min_cost": "100", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-23T23:49:49.099 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_max": "512", 2024-04-23T23:49:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_min": "64", 2024-04-23T23:49:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_degraded": "false", 2024-04-23T23:49:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_peering": "false", 2024-04-23T23:49:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_unfound": "true", 2024-04-23T23:49:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_beacon_report_interval": "300", 2024-04-23T23:49:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_duration": "30", 2024-04-23T23:49:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-23T23:49:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_max_block_size": "67108864", 2024-04-23T23:49:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-23T23:49:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_blkin_trace_all": "false", 2024-04-23T23:49:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-23T23:49:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_for_log_corruption": "false", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-23T23:49:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-23T23:49:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_update_on_start": "true", 2024-04-23T23:49:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_cap": "256", 2024-04-23T23:49:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_size_cap": "524288000", 2024-04-23T23:49:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_op_priority": "63", 2024-04-23T23:49:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_watch_timeout": "30", 2024-04-23T23:49:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_command_max_records": "256", 2024-04-23T23:49:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_compact_on_start": "false", 2024-04-23T23:49:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-23T23:49:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-23T23:49:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-23T23:49:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_on_start": "true", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_weight_set": "true", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-23T23:49:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_feed_pullee": "-1", 2024-04-23T23:49:49.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-23T23:49:49.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-23T23:49:49.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-23T23:49:49.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_misdirected_ops": "false", 2024-04-23T23:49:49.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_acting_change": "false", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_purge_strays": "false", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_op_order": "false", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-23T23:49:49.107 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_shutdown": "false", 2024-04-23T23:49:49.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-23T23:49:49.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-23T23:49:49.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_trim_objects": "false", 2024-04-23T23:49:49.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-23T23:49:49.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-23T23:49:49.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_snaps": "false", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_keys": "1024", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_stride": "524288", 2024-04-23T23:49:49.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_notify_timeout": "30", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep": "0.000000", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-23T23:49:49.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_discard_disconnected_ops": "true", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_enable_op_tracker": "true", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_info": "true", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown": "true", 2024-04-23T23:49:49.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_function_tracing": "false", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_grace": "20", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_interval": "6", 2024-04-23T23:49:49.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_peers": "10", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_size": "2000", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_stale": "600", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_max_size": "100000", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_min_size": "1000", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-23T23:49:49.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_size": "5120", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_kill_backfill_at": "0", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_cache_size": "50", 2024-04-23T23:49:49.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_dedup": "true", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max": "40", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_share_max_epochs": "40", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_name_len": "100", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_size": "0", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_backfills": "1", 2024-04-23T23:49:49.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_count": "5", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_period": "600", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_name_len": "2048", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_namespace_len": "256", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_size": "134217728", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_blocked_by": "16", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_log_entries": "10000", 2024-04-23T23:49:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-23T23:49:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pgls": "1024", 2024-04-23T23:49:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_cost": "8388608", 2024-04-23T23:49:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_objects": "10", 2024-04-23T23:49:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_scrubs": "3", 2024-04-23T23:49:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-23T23:49:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_trimming_pgs": "2", 2024-04-23T23:49:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_op_reply_len": "64", 2024-04-23T23:49:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_size": "90", 2024-04-23T23:49:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-23T23:49:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-23T23:49:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-23T23:49:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-23T23:49:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-23T23:49:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-23T23:49:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-23T23:49:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-23T23:49:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_profile": "balanced", 2024-04-23T23:49:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-23T23:49:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-23T23:49:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-23T23:49:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-23T23:49:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-23T23:49:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_base": "805306368", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_min": "134217728", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target": "4294967296", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_autotune": "false", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-23T23:49:49.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_pg_log_entries": "250", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_recovery_priority": "0", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_interval": "5", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_cache_shards": "32", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_op_tracker_shard": "32", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_auto_affinity": "true", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_node": "-1", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_prefer_iface": "true", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objecter_finishers": "1", 2024-04-23T23:49:49.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore": "bluestore", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_fuse": "false", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_tracing": "false", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_complaint_time": "30.000000", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_duration": "600", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_size": "20", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_size": "20", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_log_threshold": "5", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards": "0", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_hdd": "5", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_ssd": "8", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-23T23:49:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_min_cost": "65536", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue_cut_off": "high", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_timeout": "15", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_open_classes_on_start": "true", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_os_flags": "0", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_peering_op_priority": "255", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_cost": "1048576", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_priority": "5", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_max": "10000", 2024-04-23T23:49:49.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_min": "100", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_crimson": "false", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-23T23:49:49.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flags": "0", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_min_size": "0", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_num": "32", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pgp_num": "0", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_read_ratio": "70", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_size": "2", 2024-04-23T23:49:49.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_type": "replicated", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_push_per_object_cost": "1000", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap": "true", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_cost": "20971520", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active": "0", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_single_start": "1", 2024-04-23T23:49:49.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_priority": "3", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_priority": "5", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep": "0.000000", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_repair_during_recovery": "false", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_requested_scrub_priority": "120", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair": "false", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-23T23:49:49.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_hour": "0", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_week_day": "0", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_max": "25", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_min": "5", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_cost": "52428800", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_during_recovery": "false", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_hour": "0", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_week_day": "0", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_event_cost": "4096", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_invalid_stats": "true", 2024-04-23T23:49:49.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_preemptions": "5", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_priority": "5", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_reservation_timeout": "300000", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_sleep": "0.000000", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_skip_data_digest": "false", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_smart_report_timeout": "5", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_cost": "1048576", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_priority": "5", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-23T23:49:49.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_transaction_size": "30", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-23T23:49:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tracing": "false", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_use_stale_snap": "false", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osdc_blkin_trace_all": "false", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_kill_at": "0", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_max_join_drift": "10", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min": "500", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min_wait": "0.050000", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_propose_interval": "1.000000", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max": "500", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_min": "250", 2024-04-23T23:49:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_stash_full_interval": "25", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_max": "500", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_min": "250", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "perf": "true", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "-", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrv": "", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "public_bind_addr": "-", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network": "", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network_interface": "", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_enabled": "false", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_session_max_number": "256", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_mon_op_timeout": "0", 2024-04-23T23:49:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_osd_op_timeout": "0", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_tracing": "false", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_atime_update_interval": "60", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_parent_reads": "false", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_snap_reads": "false", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blkin_trace_all": "false", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache": "true", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-23T23:49:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_policy": "writearound", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_size": "33554432", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_clone_copy_on_read": "false", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_compression_hint": "none", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_concurrent_management_ops": "10", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_clone_format": "auto", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_data_pool": "", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_features": "61", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_format": "2", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_map_options": "", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_order": "22", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_pool": "rbd", 2024-04-23T23:49:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_count": "0", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_unit": "0", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_enable_alloc_hint": "true", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler": "simple", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-23T23:49:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_order": "24", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_pool": "", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_splay_width": "4", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_parent_reads": "false", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_snap_reads": "false", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-23T23:49:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-23T23:49:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mtime_update_interval": "60", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_non_blocking_aio": "true", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_thread_timeout": "60", 2024-04-23T23:49:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_threads": "1", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_parent_cache_enabled": "false", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_plugins": "", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst": "0", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_limit": "0", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_exclude_ops": "0", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst": "0", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_limit": "0", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-23T23:49:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_read_from_replica_policy": "default", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-23T23:49:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_skip_partial_discard": "true", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_tracing": "false", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_names": "true", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_pool": "true", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_base_url": "", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_log_level": "", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_acl_grants_max_num": "100", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_admin_entry": "admin", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-23T23:49:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_backend_store": "rados", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_barbican_url": "", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_beast_enable_async": "true", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_counters_cache": "false", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_enabled": "true", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_expiry_interval": "900", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_lru_size": "10000", 2024-04-23T23:49:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_config_store": "rados", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_content_length_compat": "false", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress": "true", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cors_rules_max_num": "100", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cross_domain_policy": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_addr": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_password": "", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-23T23:49:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_username": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_require_ssl": "true", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-23T23:49:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_addr": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_auth": "token", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_namespace": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_prefix": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_token_file": "", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_buffersize": "524288", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-23T23:49:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_time": "300", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_port": "6379", 2024-04-23T23:49:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_changes_size": "1000", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_num_shards": "128", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_window": "30", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-23T23:49:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-23T23:49:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_name": "", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_s3website_name": "", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dynamic_resharding": "true", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_gc_threads": "true", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_lc_threads": "true", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_ops_log": "false", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_quota_threads": "true", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_static_website": "false", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_usage_log": "false", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enforce_swift_acls": "true", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_exit_timeout_secs": "120", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_expose_bucket": "false", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_extended_http_attrs": "", 2024-04-23T23:49:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_filter": "none", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontends": "beast port=7480", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred": "50", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_objs": "32", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_period": "3600", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-23T23:49:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_init_timeout": "300", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_connection_idle": "300", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_domain": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password_path": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_project": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_tenant": "", 2024-04-23T23:49:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token_path": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_user": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_api_version": "2", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_domain": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_password": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_project": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_user": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_url": "", 2024-04-23T23:49:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_debug_interval": "-1", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_lock_max_time": "90", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_objs": "32", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_rules": "1000", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_worker": "3", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_thread_delay": "0", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_dnattr": "uid", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchfilter": "", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_uri": "", 2024-04-23T23:49:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_http_headers": "", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name_utc": "false", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_name_len": "0", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_size": "0", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_chunk_size": "4194304", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-23T23:49:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_listing_results": "1000", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_notify_retries": "10", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_param_size": "1048576", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_size": "5368709120", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_slo_entries": "1000", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_log_max_shards": "64", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mp_lock_max_time": "600", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-23T23:49:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_max_gc": "300", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-23T23:49:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_async_rados_threads": "32", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_control_oids": "8", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_numa_node": "-1", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_chunk_size": "100", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_gc_interval": "600", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_timeout": "600", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_tracing": "false", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_token": "", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_url": "", 2024-04-23T23:49:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_verify_ssl": "true", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_rados": "false", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_socket_path": "", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval": "2.000000", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-23T23:49:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_lanes": "3", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_partitions": "3", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-23T23:49:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_continue": "true", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_tracing": "false", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm": "", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_id": "", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region": "", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-23T23:49:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_request_uri": "", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_batch_size": "64", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_max_aio": "128", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_num_logs": "16", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_thread_interval": "600", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_resolve_cname": "false", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_run_sync_thread": "true", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-23T23:49:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_scheduler_type": "throttler", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_script_uri": "", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_service_provider_name": "", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_id": "", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_secret": "", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_entry": "sts", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_key": "sts", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_min_session_duration": "900", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_token_introspection_url": "", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_account_in_url": "false", 2024-04-23T23:49:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_entry": "auth", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_url": "", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_custom_header": "", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_need_stats": "true", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_tenant_name": "", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_token_expiration": "86400", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url": "", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url_prefix": "swift", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_lease_period": "120", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-23T23:49:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_thread_pool_size": "512", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_comment": "", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_createby": "", 2024-04-23T23:49:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_encoding": "", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_flag": "false", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_origin": "", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_tracker": "", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_trust_forwarded_https": "false", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_shards": "32", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_user_shards": "1", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_use_opa_authz": "false", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_counters_cache": "false", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-23T23:49:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_max_buckets": "1000", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_policies_max_num": "100", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_unique_email": "true", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_verify_ssl": "true", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone": "", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_id": "", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup": "", 2024-04-23T23:49:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_id": "", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rgwlc_auto_session_clear": "true", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_block_size": "4096", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_size": "536870912", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-23T23:49:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_index_type": "binary_search", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_partition_filters": "false", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_perf": "false", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "run_dir": "/var/run/ceph", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_block_create": "true", 2024-04-23T23:49:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_cache_lru_size": "67108864", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_default_max_object_size": "16777216", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_device_size": "53687091200", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_capacity": "16", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_main_device_type": "SSD", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-23T23:49:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_segment_size": "67108864", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "set_keepcaps": "false", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "setgroup": "", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser": "", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser_match_path": "", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "thp": "false", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_default_timeout": "60", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "throttler_perf_counter": "true" 2024-04-23T23:49:49.172 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:49.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-23T23:49:49.285 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2024-04-23T23:49:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T23:49:49.297 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2024-04-23T23:49:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T23:49:49.306 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2024-04-23T23:49:49.306 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T23:49:49.317 DEBUG:teuthology.orchestra.run.smithi119:> type ceph-objectstore-tool 2024-04-23T23:49:49.334 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-23T23:49:49.334 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-04-23T23:49:49.348 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-23T23:49:49.348 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-04-23T23:49:49.359 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-04-23T23:49:49.359 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-23T23:49:49.359 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-23T23:49:49.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T23:49:49.360 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-23T23:49:49.360 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-23T23:49:49.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:49:49.360 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-23T23:49:49.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:49:49.361 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-23T23:49:49.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:49:49.361 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-04-23T23:49:49.376 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:49.389+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:49.423 INFO:teuthology.orchestra.run.smithi092.stdout:mapped writes DISABLED 2024-04-23T23:49:49.424 INFO:teuthology.orchestra.run.smithi092.stdout:mapped reads DISABLED 2024-04-23T23:49:49.424 INFO:teuthology.orchestra.run.smithi092.stdout:Seed set to 7080 2024-04-23T23:49:49.465 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:49.479+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:49.513 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:49.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:49.513 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:49.513 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:49.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:49:49.538 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:49:49.538 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:49:49.555 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:49:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:49:49.562 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:49.576+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:49.572 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:49:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:49:49.590 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:49:49.591 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:49:49.609 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:49:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:49:49.627 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:49:49.627 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:49:49.644 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:49.644 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:49.644 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:49:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:49:49.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:49:49.662 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:49:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:49:49.663 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:49.676+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:49.764 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:49.778+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:49.764 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:49.764 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:49.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:49:49.865 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:49.879+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:49.918 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:49.918 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:49.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:49:49.967 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:49.980+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:50.068 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:50.081+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:50.070 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.070 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:49:50.169 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:50.182+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:50.222 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.222 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:49:50.269 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:50.282+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:49:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T23:49:50.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T23:49:50.341 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.342 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:49:50.370 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:50.384+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:50.471 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:50.484+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:50.496 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.497 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:49:50.572 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:50.585+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:50.609 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.609 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:49:50.672 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:50.685+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:50.761 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.761 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:49:50.772 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:50.786+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:50.870 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.870 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.874 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:50.886+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:50.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:49:50.974 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:50.988+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:50.981 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:50.981 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:50.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:49:51.075 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:51.088+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:51.103 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.103 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:49:51.196 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:51.209+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:51.256 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.256 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:49:51.277 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:51.291+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:51.293 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T23:49:51.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T23:49:51.398 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:51.412+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:51.409 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.409 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.418 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:49:51.479 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:51.493+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:51.520 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.521 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:49:51.581 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:51.594+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:51.631 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.631 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:49:51.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:49:51.719 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:51.733+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:51.742 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.742 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:49:51.783 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:51.796+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:51.794 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:49:51.795 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:49:51.813 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:49:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:49:51.828 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:49:51.829 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:49:51.845 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:49:51.845 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:49:51.853 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:51.853 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:51.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:49:51.862 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:49:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:49:51.879 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:49:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:49:51.883 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:51.897+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:51.896 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:49:51.897 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:49:51.913 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:49:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:49:51.963 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:51.964 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:51.964 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:49:51.964 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:49:51.964 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:51.972 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:49:51.984 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:51.998+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:52.075 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:52.075 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:49:52.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:49:52.075 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-04-23T23:49:52.075 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:52.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:49:52.085 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:52.097+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:52.186 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:49:52.187 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:52.187 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:52.200+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:52.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:49:52.287 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:52.300+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:52.296 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:52.296 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:52.296 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:49:52.296 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:49:52.297 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:52.304 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T23:49:52.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:49:52.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:49:52.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T23:49:52.402 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:52.416+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:52.410 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:52.410 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:49:52.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:49:52.410 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:49:52.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.221977645999999, 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.411 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:49:52.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:49:52.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:49:52.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.446968+0000", 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.975746259999999, 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074349669999999998, 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 70 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.446968+0000", 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.446968+0000", 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.446968+0000", 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.446971+0000", 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.446976+0000", 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9189999999999998e-06 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.446979+0000", 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0309999999999999e-06 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.448305+0000", 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013258040000000001 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.448348+0000", 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3395e-05 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.448419+0000", 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0782000000000001e-05 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.451224+0000", 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028051899999999999 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.454347+0000", 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031228940000000002 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.454380+0000", 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2699e-05 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.454403+0000", 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3189000000000001e-05 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.468723+0000", 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.953990536999999, 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066820229999999996, 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 84 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468723+0000", 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468723+0000", 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468724+0000", 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6899999999999997e-07 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468771+0000", 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6607999999999998e-05 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468783+0000", 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2138e-05 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468794+0000", 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0854e-05 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.469426+0000", 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00063184999999999999 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.469473+0000", 2024-04-23T23:49:52.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7723999999999999e-05 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.469543+0000", 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0168999999999999e-05 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.472834+0000", 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032904890000000002 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475359+0000", 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025245430000000002 2024-04-23T23:49:52.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475386+0000", 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7749000000000001e-05 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475405+0000", 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9230000000000001e-05 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.428 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:85 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.468811+0000", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.953903068000001, 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066403909999999998, 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 85 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468811+0000", 2024-04-23T23:49:52.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468811+0000", 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468812+0000", 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1900000000000002e-07 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468816+0000", 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468832+0000", 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6464e-05 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468838+0000", 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7749999999999998e-06 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.469955+0000", 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011171899999999999 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470002+0000", 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6184999999999999e-05 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470074+0000", 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2063000000000004e-05 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.472865+0000", 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027909649999999999 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475410+0000", 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025456239999999998 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475434+0000", 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3969e-05 2024-04-23T23:49:52.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475451+0000", 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7105999999999999e-05 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.468848+0000", 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.953866012000001, 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066427140000000001, 2024-04-23T23:49:52.434 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 86 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468848+0000", 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468848+0000", 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468848+0000", 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9900000000000001e-07 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468852+0000", 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.574e-06 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468857+0000", 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468860+0000", 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.14e-06 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470313+0000", 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001452135 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470367+0000", 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4744000000000003e-05 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470445+0000", 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7841000000000002e-05 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.472892+0000", 2024-04-23T23:49:52.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00244648 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475454+0000", 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002562627 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475475+0000", 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0696e-05 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.475491+0000", 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5687000000000001e-05 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.468865+0000", 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.953848771000001, 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093114870000000002, 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 87 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468865+0000", 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468865+0000", 2024-04-23T23:49:52.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468866+0000", 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-07 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468900+0000", 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468905+0000", 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0180000000000001e-06 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468909+0000", 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1099999999999996e-06 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470689+0000", 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00178086 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470736+0000", 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6437000000000003e-05 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.470810+0000", 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3901999999999998e-05 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.472922+0000", 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021117110000000001 2024-04-23T23:49:52.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.478114+0000", 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051927800000000001 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.478150+0000", 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5691000000000001e-05 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.478177+0000", 2024-04-23T23:49:52.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6709000000000001e-05 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.468920+0000", 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.953794354999999, 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093043569999999992, 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 88 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468920+0000", 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468920+0000", 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468920+0000", 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4499999999999998e-07 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468922+0000", 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2670000000000001e-06 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468927+0000", 2024-04-23T23:49:52.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1370000000000001e-06 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.468931+0000", 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.58e-06 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.472941+0000", 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004010184 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.472988+0000", 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6378000000000002e-05 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.473059+0000", 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1657999999999997e-05 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.473832+0000", 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077295300000000005 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.478182+0000", 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043501829999999997 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.478205+0000", 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2952e-05 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.478224+0000", 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.872e-05 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.598313+0000", 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.824401302, 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082830490000000007, 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 115 2024-04-23T23:49:52.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598313+0000", 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598313+0000", 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598313+0000", 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8799999999999998e-07 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598369+0000", 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5999999999999999e-05 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598382+0000", 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2486e-05 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598386+0000", 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6120000000000001e-06 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603127+0000", 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047411989999999998 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603172+0000", 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4243000000000003e-05 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603240+0000", 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8294999999999996e-05 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.604925+0000", 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001685095 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.606542+0000", 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161729 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.606572+0000", 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9861000000000001e-05 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.606596+0000", 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3580000000000001e-05 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.598396+0000", 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.824318417000001, 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010057241, 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 116 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598396+0000", 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598396+0000", 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598396+0000", 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598399+0000", 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8449999999999999e-06 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598403+0000", 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2810000000000002e-06 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598407+0000", 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0180000000000003e-06 2024-04-23T23:49:52.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603554+0000", 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005146711 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603600+0000", 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.668e-05 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603670+0000", 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9091000000000006e-05 2024-04-23T23:49:52.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.604927+0000", 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012576589999999999 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.608400+0000", 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034727289999999999 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.608429+0000", 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9196e-05 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.608453+0000", 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3665000000000001e-05 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.598414+0000", 2024-04-23T23:49:52.463 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.824299675000001, 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010120634, 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 117 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598414+0000", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598414+0000", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598415+0000", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3799999999999999e-07 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598417+0000", 2024-04-23T23:49:52.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3640000000000002e-06 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598421+0000", 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9639999999999997e-06 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.598425+0000", 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0049999999999998e-06 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603930+0000", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005505411 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.603978+0000", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7432000000000003e-05 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.604053+0000", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4962999999999994e-05 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.604929+0000", 2024-04-23T23:49:52.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00087647300000000001 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.608483+0000", 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035534540000000002 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.608513+0000", 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0629000000000001e-05 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.608535+0000", 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1701000000000001e-05 2024-04-23T23:49:52.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931480+0000", 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.491233918000001, 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068680829999999997, 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 134 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931480+0000", 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931480+0000", 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931481+0000", 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6300000000000005e-07 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931547+0000", 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6282e-05 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931555+0000", 2024-04-23T23:49:52.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8930000000000005e-06 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931561+0000", 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.75e-06 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.933980+0000", 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002419118 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.934022+0000", 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2245e-05 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.934100+0000", 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8466000000000004e-05 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938281+0000", 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004180975 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938295+0000", 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4088e-05 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938323+0000", 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7993000000000001e-05 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938348+0000", 2024-04-23T23:49:52.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4709999999999999e-05 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931575+0000", 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.491138973, 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069560029999999997, 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 135 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931575+0000", 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931575+0000", 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931576+0000", 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0100000000000005e-07 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931580+0000", 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.233e-06 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931588+0000", 2024-04-23T23:49:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6590000000000007e-06 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931593+0000", 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.934525+0000", 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002932091 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.934599+0000", 2024-04-23T23:49:52.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3726000000000005e-05 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.934720+0000", 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120764 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938391+0000", 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003671119 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938444+0000", 2024-04-23T23:49:52.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2658000000000003e-05 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938493+0000", 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8918999999999997e-05 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938531+0000", 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8439999999999998e-05 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931603+0000", 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.491110703, 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080265140000000002, 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 136 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.481 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931603+0000", 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931603+0000", 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931604+0000", 2024-04-23T23:49:52.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8299999999999997e-07 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931608+0000", 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2270000000000004e-06 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931615+0000", 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9530000000000002e-06 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931620+0000", 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5870000000000003e-06 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.935151+0000", 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003531551 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.935226+0000", 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5156999999999999e-05 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.935358+0000", 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001317 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938430+0000", 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030721630000000002 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939536+0000", 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011058089999999999 2024-04-23T23:49:52.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939588+0000", 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1820000000000002e-05 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939630+0000", 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1964e-05 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931628+0000", 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.491085783000001, 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094847089999999992, 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 137 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931628+0000", 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931628+0000", 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931629+0000", 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9299999999999998e-07 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931695+0000", 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6736000000000006e-05 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931703+0000", 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0080000000000002e-06 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931710+0000", 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4749999999999998e-06 2024-04-23T23:49:52.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.935672+0000", 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039620489999999996 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.935746+0000", 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3573999999999997e-05 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.935816+0000", 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0611000000000005e-05 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.940967+0000", 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051509160000000002 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941026+0000", 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9342999999999997e-05 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941077+0000", 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0444e-05 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941113+0000", 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6059999999999997e-05 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931721+0000", 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.490992944, 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095451370000000004, 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 138 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931721+0000", 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931721+0000", 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931722+0000", 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931727+0000", 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3229999999999997e-06 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931735+0000", 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6550000000000004e-06 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931739+0000", 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9639999999999997e-06 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938572+0000", 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068330420000000001 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938643+0000", 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1345000000000002e-05 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.938762+0000", 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118826 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.940973+0000", 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022109880000000001 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941190+0000", 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021751100000000001 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941235+0000", 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5092000000000001e-05 2024-04-23T23:49:52.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941266+0000", 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0880999999999998e-05 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931748+0000", 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.490965617000001, 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096500460000000007, 2024-04-23T23:49:52.498 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 139 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931748+0000", 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931748+0000", 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931749+0000", 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6300000000000001e-07 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931752+0000", 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8320000000000002e-06 2024-04-23T23:49:52.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931757+0000", 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2390000000000002e-06 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931762+0000", 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0669999999999999e-06 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939088+0000", 2024-04-23T23:49:52.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073260030000000002 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939158+0000", 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0059000000000001e-05 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939274+0000", 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011610799999999999 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941275+0000", 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020007929999999998 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941320+0000", 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4728999999999998e-05 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941364+0000", 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4576000000000001e-05 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941398+0000", 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4276999999999997e-05 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931772+0000", 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.490942504, 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096955979999999997, 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 140 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931772+0000", 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931772+0000", 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931772+0000", 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0200000000000002e-07 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931854+0000", 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1614999999999997e-05 2024-04-23T23:49:52.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931863+0000", 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7389999999999994e-06 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931869+0000", 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8189999999999997e-06 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939670+0000", 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078009439999999998 2024-04-23T23:49:52.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939750+0000", 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9417000000000002e-05 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.939871+0000", 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120998 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941278+0000", 2024-04-23T23:49:52.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014075629999999999 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941423+0000", 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014517499999999999 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941448+0000", 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4763999999999999e-05 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941467+0000", 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9062000000000001e-05 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.931881+0000", 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.490833241000001, 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097073860000000001, 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 141 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931881+0000", 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931881+0000", 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931881+0000", 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5599999999999995e-07 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931886+0000", 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3340000000000003e-06 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931893+0000", 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1539999999999996e-06 2024-04-23T23:49:52.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.931899+0000", 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8490000000000002e-06 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.940484+0000", 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085852240000000007 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.940566+0000", 2024-04-23T23:49:52.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1718999999999995e-05 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.940697+0000", 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000131605 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941474+0000", 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077661400000000002 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941507+0000", 2024-04-23T23:49:52.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3665999999999999e-05 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941555+0000", 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7246999999999997e-05 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.941588+0000", 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3417999999999998e-05 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:38.381491+0000", 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.041223152000001, 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006906517, 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 158 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381491+0000", 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381491+0000", 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381491+0000", 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.06e-07 2024-04-23T23:49:52.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381549+0000", 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7822999999999998e-05 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381559+0000", 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3130000000000004e-06 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381564+0000", 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1939999999999997e-06 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.383849+0000", 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002284716 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.383894+0000", 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5664999999999997e-05 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.383963+0000", 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8935000000000003e-05 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.384991+0000", 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010279149999999999 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.388350+0000", 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033593849999999999 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.388378+0000", 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7260999999999999e-05 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.388397+0000", 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9704e-05 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:49:52.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:38.381574+0000", 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.041140188, 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069103439999999997, 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 159 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:52.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381574+0000", 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381574+0000", 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381574+0000", 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9100000000000004e-07 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381579+0000", 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5619999999999997e-06 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381586+0000", 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3679999999999996e-06 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.381591+0000", 2024-04-23T23:49:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8130000000000003e-06 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.384376+0000", 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002785048 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.384421+0000", 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4359e-05 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.384492+0000", 2024-04-23T23:49:52.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1045999999999997e-05 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.385801+0000", 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001309127 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.388424+0000", 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026229809999999999 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:52.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.388451+0000", 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6829e-05 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:38.388484+0000", 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.362e-05 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:52.527 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:52.529 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:52.541+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:52.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:49:52.630 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:52.643+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:52.633 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:52.633 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:52.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:49:52.730 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:52.744+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:52.783 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:52.784 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:52.784 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:49:52.784 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:49:52.784 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:52.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:49:52.831 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:52.844+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:52.932 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:52.946+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:52.935 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:39 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'25, mlcod=17'22)", 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.417969+0000", 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.530833009, 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027752990000000002, 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.417969+0000", 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.417969+0000", 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.417970+0000", 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418049+0000", 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9141999999999995e-05 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418062+0000", 2024-04-23T23:49:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3206e-05 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418080+0000", 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7526000000000001e-05 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418122+0000", 2024-04-23T23:49:52.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2382999999999997e-05 2024-04-23T23:49:52.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418136+0000", 2024-04-23T23:49:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4457000000000001e-05 2024-04-23T23:49:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.420707+0000", 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025706100000000001 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.420745+0000", 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7512999999999998e-05 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'39)", 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.447869+0000", 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.500932807, 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003262671, 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447869+0000", 2024-04-23T23:49:52.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447869+0000", 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447870+0000", 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1299999999999999e-07 2024-04-23T23:49:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447911+0000", 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447928+0000", 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6748e-05 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447934+0000", 2024-04-23T23:49:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5610000000000004e-06 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448011+0000", 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6403999999999996e-05 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448029+0000", 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8238999999999999e-05 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.451074+0000", 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030454369999999998 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.451132+0000", 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7698999999999998e-05 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'39)", 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.448519+0000", 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.500282916, 2024-04-23T23:49:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055460409999999998, 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448519+0000", 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448519+0000", 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448520+0000", 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4200000000000001e-07 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448527+0000", 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1600000000000001e-06 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448534+0000", 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9519999999999997e-06 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448539+0000", 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448570+0000", 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1356999999999998e-05 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448583+0000", 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2721999999999999e-05 2024-04-23T23:49:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454034+0000", 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054512069999999996 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454065+0000", 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-05 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:71 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'45, mlcod=17'43)", 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.493857616, 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001853103, 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2399999999999997e-07 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454983+0000", 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7404000000000003e-05 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454995+0000", 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1982e-05 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455000+0000", 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5520000000000004e-06 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455046+0000", 2024-04-23T23:49:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5256999999999997e-05 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455065+0000", 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9921000000000001e-05 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456757+0000", 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016919350000000001 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456798+0000", 2024-04-23T23:49:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0327999999999997e-05 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:72 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'46, mlcod=17'43)", 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.455359+0000", 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.493442910000001, 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045146830000000002, 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455359+0000", 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455359+0000", 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455360+0000", 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999998e-07 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455412+0000", 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1570000000000003e-05 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455424+0000", 2024-04-23T23:49:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1960000000000001e-05 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455430+0000", 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.207e-06 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455479+0000", 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8899000000000001e-05 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455500+0000", 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1526999999999999e-05 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459794+0000", 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042935489999999998 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459874+0000", 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0267999999999998e-05 2024-04-23T23:49:52.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:73 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'47, mlcod=17'43)", 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.455689+0000", 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.493112936999999, 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004203348, 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455689+0000", 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455689+0000", 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455690+0000", 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0800000000000005e-07 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455752+0000", 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2284000000000002e-05 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455760+0000", 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7589999999999997e-06 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455764+0000", 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1389999999999997e-06 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455801+0000", 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6955000000000001e-05 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455819+0000", 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7583000000000001e-05 2024-04-23T23:49:52.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459877+0000", 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040582819999999999 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459893+0000", 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5838e-05 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:74 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'48, mlcod=17'43)", 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.492740164000001, 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003847966, 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:52.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:52.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:49:52.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456066+0000", 2024-04-23T23:49:52.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2849999999999999e-06 2024-04-23T23:49:52.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456072+0000", 2024-04-23T23:49:52.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5749999999999997e-06 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456076+0000", 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.647e-06 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456112+0000", 2024-04-23T23:49:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5611000000000002e-05 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456131+0000", 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9262999999999998e-05 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459894+0000", 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037632360000000001 2024-04-23T23:49:52.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459910+0000", 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5977e-05 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:111 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'73, mlcod=17'65)", 2024-04-23T23:49:52.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.601218+0000", 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.347583781000001, 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025268360000000002, 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601218+0000", 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601218+0000", 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601219+0000", 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9099999999999999e-07 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601235+0000", 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6337999999999998e-05 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601242+0000", 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8709999999999999e-06 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601246+0000", 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-06 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601275+0000", 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9139999999999999e-05 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601292+0000", 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.66e-05 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603717+0000", 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024254070000000001 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603745+0000", 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8368999999999999e-05 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'66)", 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.601779+0000", 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.347023043, 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001985246, 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601779+0000", 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601779+0000", 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601780+0000", 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6300000000000001e-07 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601845+0000", 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5075999999999998e-05 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601851+0000", 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601854+0000", 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.444e-06 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601885+0000", 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0499999999999999e-05 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601897+0000", 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1950000000000001e-05 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603749+0000", 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018521340000000001 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603764+0000", 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5667000000000001e-05 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'71)", 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.603360+0000", 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.345442273, 2024-04-23T23:49:52.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003021059, 2024-04-23T23:49:52.984 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603360+0000", 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603360+0000", 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603361+0000", 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1900000000000002e-07 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603408+0000", 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7018000000000003e-05 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603418+0000", 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0200000000000001e-05 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603424+0000", 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5570000000000001e-06 2024-04-23T23:49:52.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603793+0000", 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368923 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603813+0000", 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9538000000000002e-05 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.606351+0000", 2024-04-23T23:49:52.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002538111 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.606381+0000", 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0093000000000001e-05 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'71)", 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.603771+0000", 2024-04-23T23:49:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.345031194000001, 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043551129999999999, 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603771+0000", 2024-04-23T23:49:52.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603771+0000", 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603772+0000", 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603776+0000", 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6179999999999997e-06 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603787+0000", 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603793+0000", 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3820000000000003e-06 2024-04-23T23:49:52.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604251+0000", 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045776299999999997 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604268+0000", 2024-04-23T23:49:52.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7493e-05 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608090+0000", 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003822275 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608126+0000", 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5843000000000002e-05 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'71)", 2024-04-23T23:49:52.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.344673366, 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040154489999999999, 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7800000000000002e-07 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604132+0000", 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0240000000000002e-06 2024-04-23T23:49:52.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604142+0000", 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3470000000000005e-06 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:52.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604145+0000", 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604492+0000", 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000346936 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604503+0000", 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1337e-05 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608128+0000", 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003624637 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608144+0000", 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6189e-05 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'90)", 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.935419+0000", 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.013382901, 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024698939999999998, 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935419+0000", 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935419+0000", 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935420+0000", 2024-04-23T23:49:53.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935422+0000", 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.638e-06 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935429+0000", 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999999e-06 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935435+0000", 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935467+0000", 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2512999999999999e-05 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935480+0000", 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2534000000000001e-05 2024-04-23T23:49:53.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.937849+0000", 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023694699999999998 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.937889+0000", 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0147e-05 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:137 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'95, mlcod=17'90)", 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.012906705000001, 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046240889999999996, 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.65e-07 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935956+0000", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9839000000000001e-05 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935967+0000", 2024-04-23T23:49:53.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0859000000000001e-05 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935972+0000", 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2140000000000004e-06 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935999+0000", 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7630000000000001e-05 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.936009+0000", 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9860000000000006e-06 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.940483+0000", 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044731249999999997 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.940520+0000", 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7070999999999998e-05 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:155 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=17'105)", 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.566373339, 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020408430000000001, 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382508+0000", 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9160999999999996e-05 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382515+0000", 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2269999999999998e-06 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382518+0000", 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3189999999999999e-06 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382548+0000", 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0256999999999998e-05 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382560+0000", 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1739000000000001e-05 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384444+0000", 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018839499999999999 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384470+0000", 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5947000000000001e-05 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:156 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=17'105)", 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.565740286, 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032629149999999999, 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6e-07 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383065+0000", 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4930000000000001e-06 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383074+0000", 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6479999999999998e-06 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383078+0000", 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2289999999999999e-06 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383111+0000", 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2941000000000002e-05 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383122+0000", 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1647e-05 2024-04-23T23:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386302+0000", 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031794990000000001 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386325+0000", 2024-04-23T23:49:53.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3198e-05 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:157 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=17'107)", 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:53.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.565138202, 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026866809999999998, 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-07 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383669+0000", 2024-04-23T23:49:53.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8230000000000002e-06 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383675+0000", 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5439999999999998e-06 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383678+0000", 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8660000000000002e-06 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383708+0000", 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0102000000000001e-05 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383721+0000", 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.349e-05 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386327+0000", 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002605689 2024-04-23T23:49:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386351+0000", 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3873999999999999e-05 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:158 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'107)", 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.384040+0000", 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.564762049, 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042205769999999997, 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384040+0000", 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384040+0000", 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384041+0000", 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7599999999999997e-07 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384099+0000", 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8601999999999999e-05 2024-04-23T23:49:53.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384107+0000", 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3239999999999997e-06 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384111+0000", 2024-04-23T23:49:53.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384148+0000", 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6965999999999998e-05 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384166+0000", 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7941999999999999e-05 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388240+0000", 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040741700000000002 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388261+0000", 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0922999999999999e-05 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:159 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'107)", 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.384587+0000", 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.564215009, 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036990410000000001, 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384587+0000", 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384587+0000", 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384588+0000", 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1e-07 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384593+0000", 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6239999999999999e-06 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384600+0000", 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5060000000000001e-06 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384605+0000", 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3310000000000003e-06 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384633+0000", 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7892999999999999e-05 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384644+0000", 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.115e-05 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388264+0000", 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036204280000000002 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388286+0000", 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1798999999999999e-05 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:161 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'115, mlcod=17'112)", 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:53.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.559528493, 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020827720000000001, 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.41e-07 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389337+0000", 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2515999999999996e-05 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389343+0000", 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6939999999999998e-06 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389346+0000", 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389378+0000", 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1207999999999997e-05 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389390+0000", 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2534999999999999e-05 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.391325+0000", 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001935085 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.391357+0000", 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1195999999999997e-05 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.043 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:49:53.046 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:53.058+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:53.145 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:53.159+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:53.189 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:53.189 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:49:53.246 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:53.259+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:53.305 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:49:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T23:49:53.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-23T23:49:53.351 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:53.351 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:53.351 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:49:53.351 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:49:53.351 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.352 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:53.365+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:53.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:49:53.449 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:53.462+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:53.508 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:7 3.1a e24/22 3:5cea7035:::rbd_info:head v 24'1, mlcod=0'0)", 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.203027056, 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001005976, 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319448+0000", 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7959999999999998e-06 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319468+0000", 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9548999999999999e-05 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319478+0000", 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0686000000000001e-05 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319545+0000", 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6811999999999997e-05 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319578+0000", 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3247000000000002e-05 2024-04-23T23:49:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.320399+0000", 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082100599999999999 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.320447+0000", 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7658999999999999e-05 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:9 3.1a e25/22 3:5cea7035:::rbd_info:head v 25'2, mlcod=0'0)", 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:53.324049+0000", 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19841871, 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089750699999999997, 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324049+0000", 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324049+0000", 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:53.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324050+0000", 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6e-07 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324056+0000", 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8409999999999996e-06 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324065+0000", 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9870000000000005e-06 2024-04-23T23:49:53.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324073+0000", 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1019999999999997e-06 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324127+0000", 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3996999999999998e-05 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324148+0000", 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1548000000000001e-05 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324903+0000", 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075471000000000002 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:53.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324947+0000", 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4061999999999999e-05 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:53.519 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:49:53.550 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:53.563+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:53.624 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:53.624 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:49:53.676 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:53.690+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:53.736 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:53.736 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:53.737 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:49:53.737 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:49:53.737 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:49:53.786 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:53.800+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:53.852 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:53.866+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:53.889 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:53.889 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:53.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:53.889 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2024-04-23T23:49:53.889 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:53.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:49:53.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:49:53.953 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:53.967+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:54.046 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:54.047 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:54.054 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:54.068+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:54.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:49:54.061 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:49:54.061 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:49:54.079 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:49:54.079 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:49:54.096 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:49:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:49:54.113 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:49:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:49:54.132 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:49:54.132 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:49:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:49:54.150 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:49:54.155 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:54.168+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:54.159 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.159 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:49:54.169 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:49:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:49:54.186 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:49:54.187 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:49:54.255 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:54.269+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:54.290 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.290 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:49:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-04-23T23:49:54.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-23T23:49:54.356 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:54.369+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:54.416 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.416 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:49:54.456 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:54.470+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:54.558 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:54.571+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:54.568 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.568 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:49:54.659 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:54.672+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:54.682 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.682 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:49:54.759 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:54.773+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:54.790 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.790 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:49:54.860 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:54.874+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:54.900 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:54.901 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:54.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:49:54.993 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:55.007+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:55.049 INFO:teuthology.orchestra.run.smithi092.stdout:1 writesame 0x2ef5773 thru 0x2efbee5 data_size 0x1(0x6773 bytes) 2024-04-23T23:49:55.050 INFO:teuthology.orchestra.run.smithi092.stdout:2 punch from 0x25d65a8 to 0x25db349, (0x4da1 bytes) 2024-04-23T23:49:55.050 INFO:teuthology.orchestra.run.smithi092.stdout:3 write 0x22f407a thru 0x22f72cc (0x3253 bytes) 2024-04-23T23:49:55.056 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.056 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.058 INFO:teuthology.orchestra.run.smithi092.stdout:4 read 0xfd5ed thru 0xfe19c (0xbb0 bytes) 2024-04-23T23:49:55.058 INFO:teuthology.orchestra.run.smithi092.stdout:5 read 0x2015fb thru 0x20a769 (0x916f bytes) 2024-04-23T23:49:55.058 INFO:teuthology.orchestra.run.smithi092.stdout:6 read 0xc7680a thru 0xc844c3 (0xdcba bytes) 2024-04-23T23:49:55.058 INFO:teuthology.orchestra.run.smithi092.stdout:8 read 0x2e82ea7 thru 0x2e8b5bd (0x8717 bytes) 2024-04-23T23:49:55.062 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:55.076+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:55.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:49:55.066 INFO:teuthology.orchestra.run.smithi092.stdout:11 write 0x4405b6d thru 0x440bd06 (0x619a bytes) 2024-04-23T23:49:55.069 INFO:teuthology.orchestra.run.smithi092.stdout:12 write 0x3aa2c28 thru 0x3aa5dc4 (0x319d bytes) 2024-04-23T23:49:55.074 INFO:teuthology.orchestra.run.smithi092.stdout:14 read 0x380884e thru 0x3817d82 (0xf535 bytes) 2024-04-23T23:49:55.075 INFO:teuthology.orchestra.run.smithi092.stdout:15 punch from 0x1713327 to 0x1719e81, (0x6b5a bytes) 2024-04-23T23:49:55.075 INFO:teuthology.orchestra.run.smithi092.stdout:16 read 0x5dae81 thru 0x5e9a00 (0xeb80 bytes) 2024-04-23T23:49:55.075 INFO:teuthology.orchestra.run.smithi092.stdout:17 punch from 0x377250e to 0x37745c2, (0x20b4 bytes) 2024-04-23T23:49:55.095 INFO:teuthology.orchestra.run.smithi092.stdout:18 write 0xec31433 thru 0xec40334 (0xef02 bytes) 2024-04-23T23:49:55.099 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0x2097060 2024-04-23T23:49:55.114 INFO:teuthology.orchestra.run.smithi092.stdout:20 trunc from 0xec40335 to 0x2097060 2024-04-23T23:49:55.128 INFO:teuthology.orchestra.run.smithi092.stdout:21 punch from 0x1314f47 to 0x131bd85, (0x6e3e bytes) 2024-04-23T23:49:55.128 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xa288a16 2024-04-23T23:49:55.137 INFO:teuthology.orchestra.run.smithi092.stdout:23 trunc from 0x2097060 to 0xa288a16 2024-04-23T23:49:55.144 INFO:teuthology.orchestra.run.smithi092.stdout:24 writesame 0x570d418 thru 0x571b38a data_size 0x1(0xdf73 bytes) 2024-04-23T23:49:55.164 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:55.177+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:55.179 INFO:teuthology.orchestra.run.smithi092.stdout:25 writesame 0x1db9e7d thru 0x1dc638b data_size 0x1(0xc50f bytes) 2024-04-23T23:49:55.206 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xda25573 2024-04-23T23:49:55.209 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.210 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.210 INFO:teuthology.orchestra.run.smithi092.stdout:26 trunc from 0xa288a16 to 0xda25573 2024-04-23T23:49:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:27 read 0x752fe49 thru 0x7539eee (0xa0a6 bytes) 2024-04-23T23:49:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:28 read 0x5519b8 thru 0x555de1 (0x442a bytes) 2024-04-23T23:49:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:30 read 0x5fd554c thru 0x5fe4f3d (0xf9f2 bytes) 2024-04-23T23:49:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:55.218 INFO:teuthology.orchestra.run.smithi092.stdout:35 read 0xb392e46 thru 0xb3a27ec (0xf9a7 bytes) 2024-04-23T23:49:55.218 INFO:teuthology.orchestra.run.smithi092.stdout:37 read 0x7618767 thru 0x7620a78 (0x8312 bytes) 2024-04-23T23:49:55.219 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:49:55.225 INFO:teuthology.orchestra.run.smithi092.stdout:38 writesame 0xe37c1da thru 0xe3843b7 data_size 0x9(0x81de bytes) 2024-04-23T23:49:55.225 INFO:teuthology.orchestra.run.smithi092.stdout:39 write 0x39001d2 thru 0x390ac6a (0xaa99 bytes) 2024-04-23T23:49:55.228 INFO:teuthology.orchestra.run.smithi092.stdout:40 punch from 0x9707cda to 0x9708237, (0x55d bytes) 2024-04-23T23:49:55.228 INFO:teuthology.orchestra.run.smithi092.stdout:41 punch from 0x7ededa0 to 0x7ee386c, (0x4acc bytes) 2024-04-23T23:49:55.228 INFO:teuthology.orchestra.run.smithi092.stdout:43 clone 1 order 21 su 262144 sc 9 2024-04-23T23:49:55.265 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:55.279+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:55.315 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-04-23T23:49:55.323 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.323 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:49:55.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-23T23:49:55.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:49:55.372 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:55.386+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:55.466 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:55.480+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:55.475 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.475 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:49:55.514 INFO:teuthology.orchestra.run.smithi092.stdout:44 read 0x520740d thru 0x52122b9 (0xaead bytes) 2024-04-23T23:49:55.515 INFO:teuthology.orchestra.run.smithi092.stdout:45 read 0x8c799c8 thru 0x8c7b51a (0x1b53 bytes) 2024-04-23T23:49:55.515 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:55.515 INFO:teuthology.orchestra.run.smithi092.stdout:47 punch from 0x84b7c25 to 0x84bda9c, (0x5e77 bytes) 2024-04-23T23:49:55.519 INFO:teuthology.orchestra.run.smithi092.stdout:48 writesame 0x12712f0 thru 0x1280ecc data_size 0x3d(0xfbdd bytes) 2024-04-23T23:49:55.525 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:55.526 INFO:teuthology.orchestra.run.smithi092.stdout:52 clone 2 order 22 su 524288 sc 7 2024-04-23T23:49:55.568 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:55.582+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:49:55.668 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:55.682+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:55.769 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:55.783+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:55.777 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.778 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:49:55.870 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:55.884+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:55.929 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:55.929 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:55.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:49:55.982 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:55.996+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:56.041 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:56.041 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:56.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:49:56.071 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:56.085+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:56.172 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:56.186+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:56.192 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:56.192 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:56.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:49:56.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:49:56.273 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:49:56.287+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:49:56.319 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:49:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:49:56.326 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-04-23T23:49:56.326 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:49:56.342 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:49:56.343 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:49:56.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-23T23:49:56.356 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:39 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'25, mlcod=17'22)", 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.417969+0000", 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.951273793999999, 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027752990000000002, 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.417969+0000", 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.417969+0000", 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.417970+0000", 2024-04-23T23:49:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418049+0000", 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9141999999999995e-05 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418062+0000", 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3206e-05 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418080+0000", 2024-04-23T23:49:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7526000000000001e-05 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418122+0000", 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2382999999999997e-05 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.418136+0000", 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4457000000000001e-05 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.420707+0000", 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025706100000000001 2024-04-23T23:49:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.420745+0000", 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7512999999999998e-05 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'39)", 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.447869+0000", 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.921373591999998, 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003262671, 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447869+0000", 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447869+0000", 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447870+0000", 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1299999999999999e-07 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447911+0000", 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447928+0000", 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6748e-05 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.447934+0000", 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5610000000000004e-06 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448011+0000", 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6403999999999996e-05 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448029+0000", 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8238999999999999e-05 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.451074+0000", 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030454369999999998 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.451132+0000", 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7698999999999998e-05 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'39)", 2024-04-23T23:49:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.448519+0000", 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.920723701, 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055460409999999998, 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448519+0000", 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448519+0000", 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448520+0000", 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4200000000000001e-07 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448527+0000", 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1600000000000001e-06 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448534+0000", 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9519999999999997e-06 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448539+0000", 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448570+0000", 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1356999999999998e-05 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.448583+0000", 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2721999999999999e-05 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454034+0000", 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054512069999999996 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454065+0000", 2024-04-23T23:49:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-05 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:71 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'45, mlcod=17'43)", 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.914298401, 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001853103, 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454945+0000", 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2399999999999997e-07 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454983+0000", 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7404000000000003e-05 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.454995+0000", 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1982e-05 2024-04-23T23:49:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455000+0000", 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5520000000000004e-06 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455046+0000", 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5256999999999997e-05 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455065+0000", 2024-04-23T23:49:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9921000000000001e-05 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456757+0000", 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016919350000000001 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456798+0000", 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0327999999999997e-05 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:72 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'46, mlcod=17'43)", 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.455359+0000", 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.913883694999999, 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045146830000000002, 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455359+0000", 2024-04-23T23:49:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455359+0000", 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455360+0000", 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999998e-07 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455412+0000", 2024-04-23T23:49:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1570000000000003e-05 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455424+0000", 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1960000000000001e-05 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455430+0000", 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.207e-06 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455479+0000", 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8899000000000001e-05 2024-04-23T23:49:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455500+0000", 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1526999999999999e-05 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459794+0000", 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042935489999999998 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459874+0000", 2024-04-23T23:49:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0267999999999998e-05 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:73 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'47, mlcod=17'43)", 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.455689+0000", 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.913553722, 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004203348, 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455689+0000", 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455689+0000", 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455690+0000", 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0800000000000005e-07 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455752+0000", 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2284000000000002e-05 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455760+0000", 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7589999999999997e-06 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455764+0000", 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1389999999999997e-06 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455801+0000", 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6955000000000001e-05 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.455819+0000", 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7583000000000001e-05 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459877+0000", 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040582819999999999 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459893+0000", 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5838e-05 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:74 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'48, mlcod=17'43)", 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.913180949000001, 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003847966, 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456062+0000", 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456066+0000", 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2849999999999999e-06 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456072+0000", 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5749999999999997e-06 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456076+0000", 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.647e-06 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456112+0000", 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5611000000000002e-05 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.456131+0000", 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9262999999999998e-05 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459894+0000", 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037632360000000001 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.459910+0000", 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5977e-05 2024-04-23T23:49:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:111 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'73, mlcod=17'65)", 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.601218+0000", 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.768024566000001, 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025268360000000002, 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601218+0000", 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601218+0000", 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601219+0000", 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9099999999999999e-07 2024-04-23T23:49:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601235+0000", 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6337999999999998e-05 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601242+0000", 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8709999999999999e-06 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601246+0000", 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-06 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601275+0000", 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9139999999999999e-05 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601292+0000", 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.66e-05 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603717+0000", 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024254070000000001 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603745+0000", 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8368999999999999e-05 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'66)", 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.601779+0000", 2024-04-23T23:49:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.767463828, 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001985246, 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601779+0000", 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601779+0000", 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601780+0000", 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6300000000000001e-07 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601845+0000", 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5075999999999998e-05 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601851+0000", 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-04-23T23:49:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601854+0000", 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.444e-06 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601885+0000", 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0499999999999999e-05 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.601897+0000", 2024-04-23T23:49:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1950000000000001e-05 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603749+0000", 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018521340000000001 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603764+0000", 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5667000000000001e-05 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'71)", 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.603360+0000", 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.765883058, 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003021059, 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603360+0000", 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603360+0000", 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603361+0000", 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1900000000000002e-07 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603408+0000", 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7018000000000003e-05 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603418+0000", 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0200000000000001e-05 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603424+0000", 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5570000000000001e-06 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603793+0000", 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368923 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603813+0000", 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9538000000000002e-05 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.606351+0000", 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002538111 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.606381+0000", 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0093000000000001e-05 2024-04-23T23:49:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'71)", 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.603771+0000", 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.765471979000001, 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043551129999999999, 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603771+0000", 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603771+0000", 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603772+0000", 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603776+0000", 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6179999999999997e-06 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603787+0000", 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.603793+0000", 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3820000000000003e-06 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604251+0000", 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045776299999999997 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604268+0000", 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7493e-05 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608090+0000", 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003822275 2024-04-23T23:49:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608126+0000", 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5843000000000002e-05 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'71)", 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.765114150999999, 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040154489999999999, 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604129+0000", 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7800000000000002e-07 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604132+0000", 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0240000000000002e-06 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604142+0000", 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3470000000000005e-06 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604145+0000", 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604492+0000", 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000346936 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.604503+0000", 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1337e-05 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608128+0000", 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003624637 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.608144+0000", 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6189e-05 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'90)", 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.935419+0000", 2024-04-23T23:49:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.433823686, 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024698939999999998, 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935419+0000", 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935419+0000", 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935420+0000", 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935422+0000", 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.638e-06 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935429+0000", 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999999e-06 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935435+0000", 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935467+0000", 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2512999999999999e-05 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935480+0000", 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2534000000000001e-05 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.937849+0000", 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023694699999999998 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.937889+0000", 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0147e-05 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:137 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'95, mlcod=17'90)", 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.433347489999999, 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046240889999999996, 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935896+0000", 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.65e-07 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935956+0000", 2024-04-23T23:49:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9839000000000001e-05 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935967+0000", 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0859000000000001e-05 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935972+0000", 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2140000000000004e-06 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.935999+0000", 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7630000000000001e-05 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.936009+0000", 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9860000000000006e-06 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.940483+0000", 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044731249999999997 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:37.940520+0000", 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7070999999999998e-05 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:155 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=17'105)", 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.986814123999999, 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020408430000000001, 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382429+0000", 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382508+0000", 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9160999999999996e-05 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382515+0000", 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2269999999999998e-06 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382518+0000", 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3189999999999999e-06 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382548+0000", 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0256999999999998e-05 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.382560+0000", 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1739000000000001e-05 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384444+0000", 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018839499999999999 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384470+0000", 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5947000000000001e-05 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:156 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=17'105)", 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.986181071000001, 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032629149999999999, 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383062+0000", 2024-04-23T23:49:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6e-07 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383065+0000", 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4930000000000001e-06 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383074+0000", 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6479999999999998e-06 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383078+0000", 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2289999999999999e-06 2024-04-23T23:49:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383111+0000", 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2941000000000002e-05 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383122+0000", 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1647e-05 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386302+0000", 2024-04-23T23:49:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031794990000000001 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386325+0000", 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3198e-05 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:157 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=17'107)", 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.985578987, 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026866809999999998, 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383664+0000", 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-07 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383669+0000", 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8230000000000002e-06 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383675+0000", 2024-04-23T23:49:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5439999999999998e-06 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383678+0000", 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8660000000000002e-06 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383708+0000", 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0102000000000001e-05 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.383721+0000", 2024-04-23T23:49:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.349e-05 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386327+0000", 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002605689 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.386351+0000", 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3873999999999999e-05 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:158 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'107)", 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.384040+0000", 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.985202833999999, 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042205769999999997, 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384040+0000", 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384040+0000", 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384041+0000", 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7599999999999997e-07 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384099+0000", 2024-04-23T23:49:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8601999999999999e-05 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384107+0000", 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3239999999999997e-06 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384111+0000", 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384148+0000", 2024-04-23T23:49:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6965999999999998e-05 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384166+0000", 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7941999999999999e-05 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388240+0000", 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040741700000000002 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388261+0000", 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0922999999999999e-05 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:159 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'107)", 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.384587+0000", 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.984655793999998, 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036990410000000001, 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384587+0000", 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384587+0000", 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384588+0000", 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1e-07 2024-04-23T23:49:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384593+0000", 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6239999999999999e-06 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384600+0000", 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5060000000000001e-06 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384605+0000", 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3310000000000003e-06 2024-04-23T23:49:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384633+0000", 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7892999999999999e-05 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.384644+0000", 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.115e-05 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388264+0000", 2024-04-23T23:49:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036204280000000002 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.388286+0000", 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1798999999999999e-05 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4204.0:161 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'115, mlcod=17'112)", 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.979969277999999, 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020827720000000001, 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389274+0000", 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.41e-07 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389337+0000", 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2515999999999996e-05 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389343+0000", 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6939999999999998e-06 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389346+0000", 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389378+0000", 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1207999999999997e-05 2024-04-23T23:49:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.389390+0000", 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2534999999999999e-05 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.391325+0000", 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001935085 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:38.391357+0000", 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1195999999999997e-05 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.444 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:56.445 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:49:56.445 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:49:56.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:49:56.447 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:49:56.447 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:49:56.447 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone1 2024-04-23T23:49:56.447 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:49:56.448 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:49:56.448 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:49:56.448 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:49:56.449 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:56.459+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:56.471 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:49:56.471 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:49:56.491 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:49:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:49:56.547 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:56.561+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:56.557 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-04-23T23:49:56.586 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:56.586 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:56.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:49:56.647 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:56.661+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:56.748 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:49:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:49:56.748 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:56.748 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:56.761+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:56.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:49:56.849 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:56.863+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:56.856 INFO:teuthology.orchestra.run.smithi092.stdout:53 read 0xb47c9fb thru 0xb48a4a9 (0xdaaf bytes) 2024-04-23T23:49:56.857 INFO:teuthology.orchestra.run.smithi092.stdout:54 write 0x7d42585 thru 0x7d435b3 (0x102f bytes) 2024-04-23T23:49:56.864 INFO:teuthology.orchestra.run.smithi092.stdout:55 write 0x8e605b0 thru 0x8e62ba4 (0x25f5 bytes) 2024-04-23T23:49:56.867 INFO:teuthology.orchestra.run.smithi092.stdout:56 writesame 0x1d53bb2 thru 0x1d59c17 data_size 0x1b(0x6066 bytes) 2024-04-23T23:49:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:57 trunc from 0xe3843b8 to 0x94f1371 2024-04-23T23:49:56.898 INFO:teuthology.orchestra.run.smithi092.stdout:59 trunc from 0x94f1371 to 0x7c0da9c 2024-04-23T23:49:56.911 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:56.911 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:56.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:56.911 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:49:56.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.911 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:7 3.1a e24/22 3:5cea7035:::rbd_info:head v 24'1, mlcod=0'0)", 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6049207599999997, 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001005976, 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319441+0000", 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319448+0000", 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7959999999999998e-06 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319468+0000", 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9548999999999999e-05 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319478+0000", 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0686000000000001e-05 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319545+0000", 2024-04-23T23:49:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6811999999999997e-05 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.319578+0000", 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3247000000000002e-05 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.320399+0000", 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082100599999999999 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:52.320447+0000", 2024-04-23T23:49:56.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7658999999999999e-05 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:9 3.1a e25/22 3:5cea7035:::rbd_info:head v 25'2, mlcod=0'0)", 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:53.324049+0000", 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6003124140000002, 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089750699999999997, 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324049+0000", 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324049+0000", 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324050+0000", 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6e-07 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324056+0000", 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8409999999999996e-06 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324065+0000", 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9870000000000005e-06 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324073+0000", 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1019999999999997e-06 2024-04-23T23:49:56.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324127+0000", 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3996999999999998e-05 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324148+0000", 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1548000000000001e-05 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324903+0000", 2024-04-23T23:49:56.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075471000000000002 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.324947+0000", 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4061999999999999e-05 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:313 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.377689+0000", 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54667333799999995, 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001520551, 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 313 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377689+0000", 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377689+0000", 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377691+0000", 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2819999999999999e-06 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377698+0000", 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999999e-06 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377708+0000", 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2699999999999993e-06 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377718+0000", 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0674e-05 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377785+0000", 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6208e-05 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.377910+0000", 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000125368 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378061+0000", 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151521 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378871+0000", 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00080913700000000003 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379129+0000", 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025806800000000002 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379165+0000", 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6021e-05 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379209+0000", 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4382000000000003e-05 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.925 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4473.0:313 3.0 e28/22 ondisk, result = 0)", 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.379045+0000", 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54531695099999999, 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169765, 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379045+0000", 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379045+0000", 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379046+0000", 2024-04-23T23:49:56.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3500000000000005e-07 2024-04-23T23:49:56.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379051+0000", 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1909999999999999e-06 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379063+0000", 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2422999999999999e-05 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379071+0000", 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6159999999999996e-06 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379118+0000", 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7227e-05 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379125+0000", 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6789999999999999e-06 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379215+0000", 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9693999999999999e-05 2024-04-23T23:49:56.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:314 3.6 e28/22 3:663ec9c9:::rbd_header.1179d7437ec6:head v 28'1, mlcod=0'0)", 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.379765+0000", 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54459713499999995, 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069815999999999997, 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379765+0000", 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379765+0000", 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379765+0000", 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7099999999999998e-07 2024-04-23T23:49:56.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379773+0000", 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2979999999999992e-06 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379786+0000", 2024-04-23T23:49:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2592000000000001e-05 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379793+0000", 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9260000000000003e-06 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379891+0000", 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999994e-05 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379911+0000", 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0228e-05 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.380423+0000", 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00051184799999999997 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.380463+0000", 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9886999999999999e-05 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:315 3.6 e28/22 3:663ec9c9:::rbd_header.1179d7437ec6:head v 28'2, mlcod=0'0)", 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.381241+0000", 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54312043200000004, 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053920400000000005, 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381241+0000", 2024-04-23T23:49:56.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381241+0000", 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381242+0000", 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1799999999999995e-07 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381257+0000", 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5133000000000001e-05 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381267+0000", 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0047000000000001e-05 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381274+0000", 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4119999999999998e-06 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381317+0000", 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3034999999999997e-05 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381337+0000", 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0174e-05 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381741+0000", 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040420000000000001 2024-04-23T23:49:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.381781+0000", 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9585000000000001e-05 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:317 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.385550+0000", 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.53881149800000006, 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019448600000000001, 2024-04-23T23:49:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 317 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385550+0000", 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385550+0000", 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385552+0000", 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.677e-06 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385558+0000", 2024-04-23T23:49:56.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0009999999999998e-06 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385571+0000", 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3402e-05 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385578+0000", 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6189999999999996e-06 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385625+0000", 2024-04-23T23:49:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7117000000000003e-05 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385683+0000", 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7865999999999998e-05 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385745+0000", 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1804000000000007e-05 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:318 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.385841+0000", 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.53852095, 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037189599999999998, 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 318 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385841+0000", 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385841+0000", 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385841+0000", 2024-04-23T23:49:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-07 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385846+0000", 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7480000000000001e-06 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385853+0000", 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9519999999999997e-06 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385858+0000", 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7670000000000003e-06 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385898+0000", 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0361000000000002e-05 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.385946+0000", 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7493000000000001e-05 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.386213+0000", 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026706199999999999 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:329 3.6 e28/22 3:663ec9c9:::rbd_header.1179d7437ec6:head v 28'3, mlcod=28'1)", 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.390621+0000", 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.53374120400000002, 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056523499999999998, 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390621+0000", 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390621+0000", 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390622+0000", 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3680000000000001e-06 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390631+0000", 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7539999999999995e-06 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390645+0000", 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4207e-05 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390656+0000", 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1110999999999999e-05 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390712+0000", 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5748000000000002e-05 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.390733+0000", 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1146e-05 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391146+0000", 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041268300000000001 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391186+0000", 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0218e-05 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:330 3.6 e28/22 3:663ec9c9:::rbd_header.1179d7437ec6:head v 28'4, mlcod=28'2)", 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.391759+0000", 2024-04-23T23:49:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.53260313599999998, 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00051194800000000003, 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391759+0000", 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391759+0000", 2024-04-23T23:49:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391760+0000", 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7100000000000001e-07 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391766+0000", 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391779+0000", 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3192e-05 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391787+0000", 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0849999999999997e-06 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391836+0000", 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8741e-05 2024-04-23T23:49:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.391857+0000", 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0839000000000002e-05 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.392238+0000", 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038185299999999998 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.392271+0000", 2024-04-23T23:49:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2298000000000001e-05 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:339 3.6 e28/22 3:663ec9c9:::rbd_header.1179d7437ec6:head v 28'5, mlcod=28'3)", 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.396429+0000", 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.52793249900000006, 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001195651, 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396429+0000", 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396429+0000", 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396430+0000", 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:49:56.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396438+0000", 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4800000000000004e-06 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396451+0000", 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3154e-05 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396467+0000", 2024-04-23T23:49:56.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6189e-05 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396521+0000", 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3675999999999999e-05 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.396541+0000", 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0517999999999999e-05 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.397598+0000", 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00105663 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.397625+0000", 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7186e-05 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:428 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.549085+0000", 2024-04-23T23:49:56.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.37527670499999999, 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029038499999999999, 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 428 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549085+0000", 2024-04-23T23:49:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549085+0000", 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549087+0000", 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.499e-06 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549093+0000", 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9839999999999998e-06 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549108+0000", 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4861000000000001e-05 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549116+0000", 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7150000000000004e-06 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549190+0000", 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4049999999999997e-05 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549279+0000", 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8918999999999993e-05 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549376+0000", 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6356999999999999e-05 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.967 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:429 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.549470+0000", 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.37489204199999998, 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021783100000000001, 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 429 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549470+0000", 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549470+0000", 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549470+0000", 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549474+0000", 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.766e-06 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549481+0000", 2024-04-23T23:49:56.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1879999999999996e-06 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549488+0000", 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4740000000000002e-06 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549528+0000", 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0355999999999998e-05 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549583+0000", 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4564000000000001e-05 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.549688+0000", 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104962 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:453 3.6 e28/22 3:663ec9c9:::rbd_header.1179d7437ec6:head v 28'6, mlcod=28'4)", 2024-04-23T23:49:56.972 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.563635+0000", 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.36072723400000001, 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057783800000000001, 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563635+0000", 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563635+0000", 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563636+0000", 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563655+0000", 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929000000000001e-05 2024-04-23T23:49:56.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563670+0000", 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5410999999999999e-05 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563677+0000", 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5209999999999999e-06 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563733+0000", 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6963000000000003e-05 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.563758+0000", 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4159e-05 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.564183+0000", 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042548500000000002 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.564212+0000", 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9377999999999999e-05 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:477 3.19 3:9b820912:::rbd_data.11792c51182d.000000000000000a:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.626265+0000", 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.29809645699999998, 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017533699999999999, 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 477 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626265+0000", 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626265+0000", 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626267+0000", 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626275+0000", 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7060000000000004e-06 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626291+0000", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6379999999999999e-05 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626299+0000", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7370000000000006e-06 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626357+0000", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8196000000000003e-05 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.626441+0000", 2024-04-23T23:49:56.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3693e-05 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:494 3.4 3:274c9368:::rbd_data.11792c51182d.000000000000001b:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.635529+0000", 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.28883310000000001, 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031936399999999999, 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:49:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 494 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635529+0000", 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635529+0000", 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635531+0000", 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3659999999999999e-06 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635542+0000", 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0856000000000001e-05 2024-04-23T23:49:56.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635564+0000", 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.232e-05 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635592+0000", 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8136999999999998e-05 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635675+0000", 2024-04-23T23:49:56.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2712000000000004e-05 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.635848+0000", 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017297300000000001 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:513 3.0 3:05d40eee:::rbd_data.11792c51182d.000000000000002e:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:56.985 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.644729+0000", 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.27963269499999999, 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022646999999999999, 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 513 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.986 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644729+0000", 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644729+0000", 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644731+0000", 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1160000000000002e-06 2024-04-23T23:49:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644739+0000", 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1640000000000001e-06 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644755+0000", 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5695000000000001e-05 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644765+0000", 2024-04-23T23:49:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999995e-06 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644839+0000", 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3991000000000004e-05 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.644956+0000", 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116894 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:56.990 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:56.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:60 write 0x3cbaf21 thru 0x3cc333d (0x841d bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:61 read 0x55eaad3 thru 0x55f30f3 (0x8621 bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:62 write 0x769af66 thru 0x76a9113 (0xe1ae bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:63 write 0xe4e0a7f thru 0xe4ef7d7 (0xed59 bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:64 read 0xdc49b57 thru 0xdc4c1c7 (0x2671 bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:66 writesame 0x4f4e119 thru 0x4f54a5d data_size 0xd(0x6945 bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:68 writesame 0x372c641 thru 0x3737cc6 data_size 0x3d(0xb686 bytes) 2024-04-23T23:49:56.993 INFO:teuthology.orchestra.run.smithi092.stdout:69 trunc from 0xe4ef7d8 to 0x283be72 2024-04-23T23:49:56.993 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:57.004+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:56.994 INFO:teuthology.orchestra.run.smithi092.stdout:70 punch from 0x25970c9 to 0x259e367, (0x729e bytes) 2024-04-23T23:49:57.000 INFO:teuthology.orchestra.run.smithi092.stdout:71 write 0x41ce150 thru 0x41d8d06 (0xabb7 bytes) 2024-04-23T23:49:57.004 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe3a868d 2024-04-23T23:49:57.016 INFO:teuthology.orchestra.run.smithi092.stdout:72 trunc from 0x41d8d07 to 0xe3a868d 2024-04-23T23:49:57.026 INFO:teuthology.orchestra.run.smithi092.stdout:75 trunc from 0xe3a868d to 0x9e3236d 2024-04-23T23:49:57.033 INFO:teuthology.orchestra.run.smithi092.stdout:76 writesame 0x109aead thru 0x10a6122 data_size 0x2(0xb276 bytes) 2024-04-23T23:49:57.049 INFO:teuthology.orchestra.run.smithi092.stdout:77 read 0x4a059e4 thru 0x4a14003 (0xe620 bytes) 2024-04-23T23:49:57.062 INFO:teuthology.orchestra.run.smithi092.stdout:78 writesame 0xdf07e82 thru 0xdf09446 data_size 0x1(0x15c5 bytes) 2024-04-23T23:49:57.062 INFO:teuthology.orchestra.run.smithi092.stdout:79 clone 3 order 24 su 262144 sc 9 2024-04-23T23:49:57.091 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:57.105+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:57.106 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:57.107 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:57.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:49:57.192 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:57.206+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:57.258 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:49:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:49:57.258 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:57.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:49:57.293 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:57.307+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-04-23T23:49:57.341 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T23:49:57.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:49:57.378 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:313 3.0 e28/22 3:04479396:::rbd_id.image_client.0-clone2:head v 28'1, mlcod=0'0)", 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.014589435, 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082763200000000002, 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.91e-07 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378186+0000", 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1020000000000002e-06 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378197+0000", 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.061e-05 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378206+0000", 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1400000000000006e-06 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378280+0000", 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3749000000000002e-05 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378307+0000", 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7475999999999999e-05 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378954+0000", 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064691200000000003 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379008+0000", 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3352e-05 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:536 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'1, mlcod=0'0)", 2024-04-23T23:49:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.881317+0000", 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.51145285299999999, 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00077659799999999998, 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881317+0000", 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881317+0000", 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881318+0000", 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.288e-06 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881347+0000", 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8812000000000001e-05 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881364+0000", 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7028999999999999e-05 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881374+0000", 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7610000000000002e-06 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881432+0000", 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8906999999999997e-05 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881453+0000", 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0421e-05 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.882062+0000", 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060873000000000003 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.882093+0000", 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1649999999999997e-05 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:541 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'2, mlcod=0'0)", 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.894384+0000", 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.49838582300000001, 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071852400000000001, 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894384+0000", 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894384+0000", 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894385+0000", 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894392+0000", 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5900000000000002e-06 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894406+0000", 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4062e-05 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894418+0000", 2024-04-23T23:49:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1841000000000001e-05 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894488+0000", 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9948000000000002e-05 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894508+0000", 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9531999999999999e-05 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895055+0000", 2024-04-23T23:49:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054707200000000001 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895102+0000", 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7401999999999998e-05 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:566 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'3, mlcod=28'1)", 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.47724258000000003, 2024-04-23T23:49:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001175798, 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915528+0000", 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0580000000000001e-06 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915539+0000", 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638999999999999e-05 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915554+0000", 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5141e-05 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915565+0000", 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0789e-05 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915620+0000", 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5189999999999998e-05 2024-04-23T23:49:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915642+0000", 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2262000000000001e-05 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916672+0000", 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010296109999999999 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916703+0000", 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1108000000000001e-05 2024-04-23T23:49:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.397 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:57.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:49:57.399 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone2 intact 2024-04-23T23:49:57.399 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:57.412+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:57.499 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:57.513+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:57.551 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:57.552 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:57.552 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:49:57.552 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:57.553 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-04-23T23:49:57.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:49:57.600 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:57.614+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:57.656 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:49:57.673 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-23T23:49:57.359955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1209060,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":249,"num_write":205,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219376,"kb_used_data":3188,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755344,"statfs":{"total":773094113280,"available":772869472256,"internally_reserved":0,"allocated":3264512,"data_stored":1934036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":618692,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":200,"num_read_kb":149,"num_write":46,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000563"},"pg_stats":[{"pgid":"3.1f","version":"26'3","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.534751+0000","last_change":"2024-04-23T23:49:53.323402+0000","last_active":"2024-04-23T23:49:55.534751+0000","last_peered":"2024-04-23T23:49:55.534751+0000","last_clean":"2024-04-23T23:49:55.534751+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:49:55.534751+0000","last_undegraded":"2024-04-23T23:49:55.534751+0000","last_fullsized":"2024-04-23T23:49:55.534751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:27.709138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085782299999999996,"stat_sum":{"num_bytes":618608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":3,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337848+0000","last_change":"2024-04-23T23:49:53.334943+0000","last_active":"2024-04-23T23:49:55.337848+0000","last_peered":"2024-04-23T23:49:55.337848+0000","last_clean":"2024-04-23T23:49:55.337848+0000","last_became_active":"2024-04-23T23:49:51.319750+0000","last_became_peered":"2024-04-23T23:49:51.319750+0000","last_unstale":"2024-04-23T23:49:55.337848+0000","last_undegraded":"2024-04-23T23:49:55.337848+0000","last_fullsized":"2024-04-23T23:49:55.337848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:29.453087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067565599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.328510+0000","last_change":"2024-04-23T23:49:53.328510+0000","last_active":"2024-04-23T23:49:53.328510+0000","last_peered":"2024-04-23T23:49:53.328510+0000","last_clean":"2024-04-23T23:49:53.328510+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:49:53.328510+0000","last_undegraded":"2024-04-23T23:49:53.328510+0000","last_fullsized":"2024-04-23T23:49:53.328510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:30:50.310984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018779,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"25'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.329002+0000","last_change":"2024-04-23T23:49:53.328456+0000","last_active":"2024-04-23T23:49:53.329002+0000","last_peered":"2024-04-23T23:49:53.329002+0000","last_clean":"2024-04-23T23:49:53.329002+0000","last_became_active":"2024-04-23T23:49:51.317767+0000","last_became_peered":"2024-04-23T23:49:51.317767+0000","last_unstale":"2024-04-23T23:49:53.329002+0000","last_undegraded":"2024-04-23T23:49:53.329002+0000","last_fullsized":"2024-04-23T23:49:53.329002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:30.222812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099769099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"27'7","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.541051+0000","last_change":"2024-04-23T23:49:53.322810+0000","last_active":"2024-04-23T23:49:55.541051+0000","last_peered":"2024-04-23T23:49:55.541051+0000","last_clean":"2024-04-23T23:49:55.541051+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:49:55.541051+0000","last_undegraded":"2024-04-23T23:49:55.541051+0000","last_fullsized":"2024-04-23T23:49:55.541051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:16:39.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079418399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":53,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"25'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324072+0000","last_change":"2024-04-23T23:49:53.322527+0000","last_active":"2024-04-23T23:49:54.324072+0000","last_peered":"2024-04-23T23:49:54.324072+0000","last_clean":"2024-04-23T23:49:54.324072+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:49:54.324072+0000","last_undegraded":"2024-04-23T23:49:54.324072+0000","last_fullsized":"2024-04-23T23:49:54.324072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:39.253436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041081400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468788+0000","last_change":"2024-04-23T23:49:53.327322+0000","last_active":"2024-04-23T23:49:55.468788+0000","last_peered":"2024-04-23T23:49:55.468788+0000","last_clean":"2024-04-23T23:49:55.468788+0000","last_became_active":"2024-04-23T23:49:51.321539+0000","last_became_peered":"2024-04-23T23:49:51.321539+0000","last_unstale":"2024-04-23T23:49:55.468788+0000","last_undegraded":"2024-04-23T23:49:55.468788+0000","last_fullsized":"2024-04-23T23:49:55.468788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:15.545560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038598300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"26'2","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337724+0000","last_change":"2024-04-23T23:49:53.333734+0000","last_active":"2024-04-23T23:49:55.337724+0000","last_peered":"2024-04-23T23:49:55.337724+0000","last_clean":"2024-04-23T23:49:55.337724+0000","last_became_active":"2024-04-23T23:49:51.321251+0000","last_became_peered":"2024-04-23T23:49:51.321251+0000","last_unstale":"2024-04-23T23:49:55.337724+0000","last_undegraded":"2024-04-23T23:49:55.337724+0000","last_fullsized":"2024-04-23T23:49:55.337724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:13.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064860299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328796+0000","last_change":"2024-04-23T23:49:53.323222+0000","last_active":"2024-04-23T23:49:55.328796+0000","last_peered":"2024-04-23T23:49:55.328796+0000","last_clean":"2024-04-23T23:49:55.328796+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:49:55.328796+0000","last_undegraded":"2024-04-23T23:49:55.328796+0000","last_fullsized":"2024-04-23T23:49:55.328796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:26:05.273173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091814399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.327873+0000","last_change":"2024-04-23T23:49:53.327873+0000","last_active":"2024-04-23T23:49:53.327873+0000","last_peered":"2024-04-23T23:49:53.327873+0000","last_clean":"2024-04-23T23:49:53.327873+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:49:53.327873+0000","last_undegraded":"2024-04-23T23:49:53.327873+0000","last_fullsized":"2024-04-23T23:49:53.327873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:12:57.310031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039462200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328091+0000","last_change":"2024-04-23T23:49:53.781187+0000","last_active":"2024-04-23T23:49:54.328091+0000","last_peered":"2024-04-23T23:49:54.328091+0000","last_clean":"2024-04-23T23:49:54.328091+0000","last_became_active":"2024-04-23T23:49:51.321623+0000","last_became_peered":"2024-04-23T23:49:51.321623+0000","last_unstale":"2024-04-23T23:49:54.328091+0000","last_undegraded":"2024-04-23T23:49:54.328091+0000","last_fullsized":"2024-04-23T23:49:54.328091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:47:06.585826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066676300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324344+0000","last_change":"2024-04-23T23:49:53.322838+0000","last_active":"2024-04-23T23:49:54.324344+0000","last_peered":"2024-04-23T23:49:54.324344+0000","last_clean":"2024-04-23T23:49:54.324344+0000","last_became_active":"2024-04-23T23:49:51.320365+0000","last_became_peered":"2024-04-23T23:49:51.320365+0000","last_unstale":"2024-04-23T23:49:54.324344+0000","last_undegraded":"2024-04-23T23:49:54.324344+0000","last_fullsized":"2024-04-23T23:49:54.324344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:04.801819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042792500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.335655+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:49:54.335655+0000","last_peered":"2024-04-23T23:49:54.335655+0000","last_clean":"2024-04-23T23:49:54.335655+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:49:54.335655+0000","last_undegraded":"2024-04-23T23:49:54.335655+0000","last_fullsized":"2024-04-23T23:49:54.335655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328797+0000","last_change":"2024-04-23T23:49:53.322476+0000","last_active":"2024-04-23T23:49:55.328797+0000","last_peered":"2024-04-23T23:49:55.328797+0000","last_clean":"2024-04-23T23:49:55.328797+0000","last_became_active":"2024-04-23T23:49:51.317534+0000","last_became_peered":"2024-04-23T23:49:51.317534+0000","last_unstale":"2024-04-23T23:49:55.328797+0000","last_undegraded":"2024-04-23T23:49:55.328797+0000","last_fullsized":"2024-04-23T23:49:55.328797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:10.088753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046868700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468960+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:49:55.468960+0000","last_peered":"2024-04-23T23:49:55.468960+0000","last_clean":"2024-04-23T23:49:55.468960+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:49:55.468960+0000","last_undegraded":"2024-04-23T23:49:55.468960+0000","last_fullsized":"2024-04-23T23:49:55.468960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337896+0000","last_change":"2024-04-23T23:49:53.334802+0000","last_active":"2024-04-23T23:49:55.337896+0000","last_peered":"2024-04-23T23:49:55.337896+0000","last_clean":"2024-04-23T23:49:55.337896+0000","last_became_active":"2024-04-23T23:49:51.320498+0000","last_became_peered":"2024-04-23T23:49:51.320498+0000","last_unstale":"2024-04-23T23:49:55.337896+0000","last_undegraded":"2024-04-23T23:49:55.337896+0000","last_fullsized":"2024-04-23T23:49:55.337896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:04:31.101139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010602859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328187+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:49:54.328187+0000","last_peered":"2024-04-23T23:49:54.328187+0000","last_clean":"2024-04-23T23:49:54.328187+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:49:54.328187+0000","last_undegraded":"2024-04-23T23:49:54.328187+0000","last_fullsized":"2024-04-23T23:49:54.328187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468979+0000","last_change":"2024-04-23T23:49:53.327533+0000","last_active":"2024-04-23T23:49:55.468979+0000","last_peered":"2024-04-23T23:49:55.468979+0000","last_clean":"2024-04-23T23:49:55.468979+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:49:55.468979+0000","last_undegraded":"2024-04-23T23:49:55.468979+0000","last_fullsized":"2024-04-23T23:49:55.468979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:32.879558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056687800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.469105+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:49:55.469105+0000","last_peered":"2024-04-23T23:49:55.469105+0000","last_clean":"2024-04-23T23:49:55.469105+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:49:55.469105+0000","last_undegraded":"2024-04-23T23:49:55.469105+0000","last_fullsized":"2024-04-23T23:49:55.469105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'2","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.345808+0000","last_change":"2024-04-23T23:49:53.328813+0000","last_active":"2024-04-23T23:49:53.345808+0000","last_peered":"2024-04-23T23:49:53.345808+0000","last_clean":"2024-04-23T23:49:53.345808+0000","last_became_active":"2024-04-23T23:49:51.319410+0000","last_became_peered":"2024-04-23T23:49:51.319410+0000","last_unstale":"2024-04-23T23:49:53.345808+0000","last_undegraded":"2024-04-23T23:49:53.345808+0000","last_fullsized":"2024-04-23T23:49:53.345808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:22.816805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154289,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324216+0000","last_change":"2024-04-23T23:49:53.322781+0000","last_active":"2024-04-23T23:49:54.324216+0000","last_peered":"2024-04-23T23:49:54.324216+0000","last_clean":"2024-04-23T23:49:54.324216+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:49:54.324216+0000","last_undegraded":"2024-04-23T23:49:54.324216+0000","last_fullsized":"2024-04-23T23:49:54.324216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:50:09.670335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054288899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.327325+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:49:53.327325+0000","last_peered":"2024-04-23T23:49:53.327325+0000","last_clean":"2024-04-23T23:49:53.327325+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:49:53.327325+0000","last_undegraded":"2024-04-23T23:49:53.327325+0000","last_fullsized":"2024-04-23T23:49:53.327325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:39.497845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468795+0000","last_change":"2024-04-23T23:49:53.327404+0000","last_active":"2024-04-23T23:49:55.468795+0000","last_peered":"2024-04-23T23:49:55.468795+0000","last_clean":"2024-04-23T23:49:55.468795+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:49:55.468795+0000","last_undegraded":"2024-04-23T23:49:55.468795+0000","last_fullsized":"2024-04-23T23:49:55.468795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:16.224406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043824499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.322309+0000","last_change":"2024-04-23T23:49:43.280465+0000","last_active":"2024-04-23T23:49:53.322309+0000","last_peered":"2024-04-23T23:49:53.322309+0000","last_clean":"2024-04-23T23:49:53.322309+0000","last_became_active":"2024-04-23T23:49:43.280144+0000","last_became_peered":"2024-04-23T23:49:43.280144+0000","last_unstale":"2024-04-23T23:49:53.322309+0000","last_undegraded":"2024-04-23T23:49:53.322309+0000","last_fullsized":"2024-04-23T23:49:53.322309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:42:45.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328608+0000","last_change":"2024-04-23T23:49:53.322700+0000","last_active":"2024-04-23T23:49:55.328608+0000","last_peered":"2024-04-23T23:49:55.328608+0000","last_clean":"2024-04-23T23:49:55.328608+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:49:55.328608+0000","last_undegraded":"2024-04-23T23:49:55.328608+0000","last_fullsized":"2024-04-23T23:49:55.328608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:58.702424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085923399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328150+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:49:54.328150+0000","last_peered":"2024-04-23T23:49:54.328150+0000","last_clean":"2024-04-23T23:49:54.328150+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:49:54.328150+0000","last_undegraded":"2024-04-23T23:49:54.328150+0000","last_fullsized":"2024-04-23T23:49:54.328150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:28:10.960350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"27'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.517355+0000","last_change":"2024-04-23T23:49:53.334574+0000","last_active":"2024-04-23T23:49:55.517355+0000","last_peered":"2024-04-23T23:49:55.517355+0000","last_clean":"2024-04-23T23:49:55.517355+0000","last_became_active":"2024-04-23T23:49:51.318301+0000","last_became_peered":"2024-04-23T23:49:51.318301+0000","last_unstale":"2024-04-23T23:49:55.517355+0000","last_undegraded":"2024-04-23T23:49:55.517355+0000","last_fullsized":"2024-04-23T23:49:55.517355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:19.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011802519999999999,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.327317+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:49:53.327317+0000","last_peered":"2024-04-23T23:49:53.327317+0000","last_clean":"2024-04-23T23:49:53.327317+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:49:53.327317+0000","last_undegraded":"2024-04-23T23:49:53.327317+0000","last_fullsized":"2024-04-23T23:49:53.327317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328212+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:49:54.328212+0000","last_peered":"2024-04-23T23:49:54.328212+0000","last_clean":"2024-04-23T23:49:54.328212+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:49:54.328212+0000","last_undegraded":"2024-04-23T23:49:54.328212+0000","last_fullsized":"2024-04-23T23:49:54.328212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337712+0000","last_change":"2024-04-23T23:49:53.333756+0000","last_active":"2024-04-23T23:49:55.337712+0000","last_peered":"2024-04-23T23:49:55.337712+0000","last_clean":"2024-04-23T23:49:55.337712+0000","last_became_active":"2024-04-23T23:49:51.317708+0000","last_became_peered":"2024-04-23T23:49:51.317708+0000","last_unstale":"2024-04-23T23:49:55.337712+0000","last_undegraded":"2024-04-23T23:49:55.337712+0000","last_fullsized":"2024-04-23T23:49:55.337712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:09:35.137542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041896000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324272+0000","last_change":"2024-04-23T23:49:53.322582+0000","last_active":"2024-04-23T23:49:54.324272+0000","last_peered":"2024-04-23T23:49:54.324272+0000","last_clean":"2024-04-23T23:49:54.324272+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:49:54.324272+0000","last_undegraded":"2024-04-23T23:49:54.324272+0000","last_fullsized":"2024-04-23T23:49:54.324272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:56:44.873026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045422700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.335720+0000","last_change":"2024-04-23T23:49:53.494617+0000","last_active":"2024-04-23T23:49:54.335720+0000","last_peered":"2024-04-23T23:49:54.335720+0000","last_clean":"2024-04-23T23:49:54.335720+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:49:54.335720+0000","last_undegraded":"2024-04-23T23:49:54.335720+0000","last_fullsized":"2024-04-23T23:49:54.335720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:39.495902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039630999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"27'22","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.340620+0000","last_change":"2024-04-23T23:49:53.334695+0000","last_active":"2024-04-23T23:49:55.340620+0000","last_peered":"2024-04-23T23:49:55.340620+0000","last_clean":"2024-04-23T23:49:55.340620+0000","last_became_active":"2024-04-23T23:49:51.317643+0000","last_became_peered":"2024-04-23T23:49:51.317643+0000","last_unstale":"2024-04-23T23:49:55.340620+0000","last_undegraded":"2024-04-23T23:49:55.340620+0000","last_fullsized":"2024-04-23T23:49:55.340620+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:26.033458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001287243,"stat_sum":{"num_bytes":49,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":90,"num_read_kb":68,"num_write":31,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.328808+0000","last_change":"2024-04-23T23:49:53.328808+0000","last_active":"2024-04-23T23:49:53.328808+0000","last_peered":"2024-04-23T23:49:53.328808+0000","last_clean":"2024-04-23T23:49:53.328808+0000","last_became_active":"2024-04-23T23:49:51.319004+0000","last_became_peered":"2024-04-23T23:49:51.319004+0000","last_unstale":"2024-04-23T23:49:53.328808+0000","last_undegraded":"2024-04-23T23:49:53.328808+0000","last_fullsized":"2024-04-23T23:49:53.328808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:33:26.486284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012607440000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328162+0000","last_change":"2024-04-23T23:49:53.781509+0000","last_active":"2024-04-23T23:49:54.328162+0000","last_peered":"2024-04-23T23:49:54.328162+0000","last_clean":"2024-04-23T23:49:54.328162+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:49:54.328162+0000","last_undegraded":"2024-04-23T23:49:54.328162+0000","last_fullsized":"2024-04-23T23:49:54.328162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:55.322236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090598199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328220+0000","last_change":"2024-04-23T23:49:53.781245+0000","last_active":"2024-04-23T23:49:54.328220+0000","last_peered":"2024-04-23T23:49:54.328220+0000","last_clean":"2024-04-23T23:49:54.328220+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:49:54.328220+0000","last_undegraded":"2024-04-23T23:49:54.328220+0000","last_fullsized":"2024-04-23T23:49:54.328220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:09:58.407504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071364900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328329+0000","last_change":"2024-04-23T23:49:53.781604+0000","last_active":"2024-04-23T23:49:54.328329+0000","last_peered":"2024-04-23T23:49:54.328329+0000","last_clean":"2024-04-23T23:49:54.328329+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:49:54.328329+0000","last_undegraded":"2024-04-23T23:49:54.328329+0000","last_fullsized":"2024-04-23T23:49:54.328329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:01.557911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097850600000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.335740+0000","last_change":"2024-04-23T23:49:53.494621+0000","last_active":"2024-04-23T23:49:54.335740+0000","last_peered":"2024-04-23T23:49:54.335740+0000","last_clean":"2024-04-23T23:49:54.335740+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:49:54.335740+0000","last_undegraded":"2024-04-23T23:49:54.335740+0000","last_fullsized":"2024-04-23T23:49:54.335740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:42.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400901,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328077+0000","last_change":"2024-04-23T23:49:53.780895+0000","last_active":"2024-04-23T23:49:54.328077+0000","last_peered":"2024-04-23T23:49:54.328077+0000","last_clean":"2024-04-23T23:49:54.328077+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:49:54.328077+0000","last_undegraded":"2024-04-23T23:49:54.328077+0000","last_fullsized":"2024-04-23T23:49:54.328077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:17:01.882241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049304299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328575+0000","last_change":"2024-04-23T23:49:53.323023+0000","last_active":"2024-04-23T23:49:55.328575+0000","last_peered":"2024-04-23T23:49:55.328575+0000","last_clean":"2024-04-23T23:49:55.328575+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:49:55.328575+0000","last_undegraded":"2024-04-23T23:49:55.328575+0000","last_fullsized":"2024-04-23T23:49:55.328575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:58.675393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000902432,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328023+0000","last_change":"2024-04-23T23:49:53.780967+0000","last_active":"2024-04-23T23:49:54.328023+0000","last_peered":"2024-04-23T23:49:54.328023+0000","last_clean":"2024-04-23T23:49:54.328023+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:49:54.328023+0000","last_undegraded":"2024-04-23T23:49:54.328023+0000","last_fullsized":"2024-04-23T23:49:54.328023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:43.662867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053415600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":618692,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":200,"num_read_kb":149,"num_write":46,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":100823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":94988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":685383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":685337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":88766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":94985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":94988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":88766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:49:57.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:49:57.684 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:49:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:49:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:49:57.684 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:57.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:49:57.701 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:57.715+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:57.802 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:57.816+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:57.850 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:49:57.850 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:12 3.5 e25/22 3:a49bdf6f:::rbd_header.11792c51182d:head v 25'1, mlcod=0'0)", 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:53.331348+0000", 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5317590020000003, 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012065400000000001, 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331348+0000", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331348+0000", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331348+0000", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.04e-07 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331364+0000", 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5736e-05 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331375+0000", 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0560000000000001e-05 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331384+0000", 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8220000000000002e-06 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331440+0000", 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5646999999999998e-05 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.331469+0000", 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8504999999999999e-05 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.332519+0000", 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001049983 2024-04-23T23:49:57.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.332555+0000", 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6083e-05 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:13 3.b 3:d1a1a168:::rbd_object_map.11792c51182d:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:53.333643+0000", 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5294639960000005, 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026818580000000001, 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333643+0000", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333643+0000", 2024-04-23T23:49:57.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333645+0000", 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.722e-06 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333652+0000", 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:49:57.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333663+0000", 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0288e-05 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333672+0000", 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.825e-06 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333729+0000", 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6171000000000001e-05 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.333865+0000", 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013599 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.334082+0000", 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021703200000000001 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.336156+0000", 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020748350000000001 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.336216+0000", 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9179999999999999e-05 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.336277+0000", 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1416999999999998e-05 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:53.336325+0000", 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7994000000000002e-05 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:398 3.15 e28/22 3:a80810de:::rbd_object_map.1179e4f9b547:head v 28'6, mlcod=27'4)", 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.404940+0000", 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.458166898, 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00094401999999999995, 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.404940+0000", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.404940+0000", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.404941+0000", 2024-04-23T23:49:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999999e-07 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.404950+0000", 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8429999999999999e-06 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.404976+0000", 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5732e-05 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.404998+0000", 2024-04-23T23:49:57.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2731e-05 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.405069+0000", 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0074999999999996e-05 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.405093+0000", 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.44e-05 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.405848+0000", 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075536899999999998 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.405884+0000", 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5908999999999998e-05 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:410 3.15 e28/22 3:a80810de:::rbd_object_map.1179e4f9b547:head v 28'7, mlcod=28'5)", 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.406932+0000", 2024-04-23T23:49:57.865 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4561748640000001, 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017288379999999999, 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.406932+0000", 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.406932+0000", 2024-04-23T23:49:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.406933+0000", 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.406940+0000", 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2289999999999997e-06 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.406950+0000", 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7890000000000006e-06 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.406960+0000", 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0169999999999999e-05 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.407023+0000", 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2452000000000004e-05 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.407046+0000", 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3082e-05 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.408612+0000", 2024-04-23T23:49:57.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001566273 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.408661+0000", 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9243999999999998e-05 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.870 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:532 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'3, mlcod=28'1)", 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.875960+0000", 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.98714672299999995, 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017652239999999999, 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.875960+0000", 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.875960+0000", 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.875969+0000", 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2490000000000001e-06 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.875974+0000", 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9339999999999997e-06 2024-04-23T23:49:57.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.875983+0000", 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0159999999999997e-06 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.875987+0000", 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7079999999999996e-06 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.876022+0000", 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4953999999999998e-05 2024-04-23T23:49:57.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.876035+0000", 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656999999999999e-05 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.877684+0000", 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016492060000000001 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.877726+0000", 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1499999999999999e-05 2024-04-23T23:49:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:539 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'6, mlcod=28'4)", 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.894364+0000", 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.968743032, 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010350089999999999, 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894364+0000", 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894364+0000", 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894365+0000", 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.95e-07 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894373+0000", 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9540000000000004e-06 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894389+0000", 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5767e-05 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894396+0000", 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8510000000000001e-06 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894460+0000", 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4177999999999994e-05 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894485+0000", 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5556000000000001e-05 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895364+0000", 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087907800000000004 2024-04-23T23:49:57.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895399+0000", 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4629999999999999e-05 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:542 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'8, mlcod=28'4)", 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.895648+0000", 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.96745874099999996, 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001002492, 2024-04-23T23:49:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895648+0000", 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895648+0000", 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895649+0000", 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895652+0000", 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3699999999999999e-06 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895666+0000", 2024-04-23T23:49:57.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4273999999999999e-05 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895678+0000", 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1443999999999999e-05 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895715+0000", 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7389999999999999e-05 2024-04-23T23:49:57.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895727+0000", 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1674e-05 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.896620+0000", 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089317900000000002 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.896651+0000", 2024-04-23T23:49:57.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0786e-05 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:573 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'18, mlcod=28'13)", 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.923959+0000", 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.939148494, 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00268461, 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.923959+0000", 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.923959+0000", 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.923959+0000", 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.65e-07 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.923969+0000", 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0202e-05 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.923976+0000", 2024-04-23T23:49:57.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.923981+0000", 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924011+0000", 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0590999999999999e-05 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924022+0000", 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0659000000000001e-05 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.926603+0000", 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002581411 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.926643+0000", 2024-04-23T23:49:57.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9851999999999997e-05 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:575 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'19, mlcod=28'13)", 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.924762+0000", 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.93834479500000001, 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022440379999999998, 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924762+0000", 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924762+0000", 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924763+0000", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6700000000000003e-07 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924767+0000", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0879999999999997e-06 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924777+0000", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924781+0000", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5619999999999997e-06 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924828+0000", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6912000000000001e-05 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.924842+0000", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3733999999999999e-05 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.926957+0000", 2024-04-23T23:49:57.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021155950000000001 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.927006+0000", 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8900999999999998e-05 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:576 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'20, mlcod=28'13)", 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.925576+0000", 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.93753124099999996, 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014618910000000001, 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925576+0000", 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925576+0000", 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925577+0000", 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4900000000000005e-07 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925583+0000", 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4230000000000002e-06 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925596+0000", 2024-04-23T23:49:57.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2928000000000001e-05 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925608+0000", 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2123999999999999e-05 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925665+0000", 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6736e-05 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.925679+0000", 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4139e-05 2024-04-23T23:49:57.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.927010+0000", 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013311060000000001 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.927038+0000", 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7685999999999999e-05 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:612 3.1e 3:7cb8dc7c:::rbd_data.1179d7437ec6.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3465497~26949] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.961916+0000", 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.90119119000000003, 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016195840000000001, 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:57.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 612 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.961916+0000", 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.961916+0000", 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.961917+0000", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4640000000000001e-06 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.961933+0000", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5814000000000001e-05 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.961945+0000", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1430000000000001e-05 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.961976+0000", 2024-04-23T23:49:57.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0988999999999998e-05 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.962032+0000", 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6607000000000002e-05 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.962118+0000", 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5983000000000007e-05 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.962233+0000", 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114966 2024-04-23T23:49:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.963076+0000", 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084287499999999998 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.963478+0000", 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000401996 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.963509+0000", 2024-04-23T23:49:57.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0765000000000001e-05 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.963536+0000", 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6695000000000001e-05 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:615 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'26, mlcod=28'24)", 2024-04-23T23:49:57.903 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.964948+0000", 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.898159135, 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00094970600000000003, 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.964948+0000", 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.964948+0000", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.964950+0000", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5260000000000001e-06 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.964960+0000", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0834e-05 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.964995+0000", 2024-04-23T23:49:57.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4582000000000002e-05 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.965012+0000", 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6679000000000001e-05 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.965118+0000", 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106074 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.965154+0000", 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6463e-05 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.965866+0000", 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071142799999999995 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.965898+0000", 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2119999999999997e-05 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:619 3.1e 3:7cb8dc7c:::rbd_data.1179d7437ec6.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.984371+0000", 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.87873624900000002, 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017627770000000001, 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 619 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984371+0000", 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984371+0000", 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984372+0000", 2024-04-23T23:49:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.274e-06 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984378+0000", 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7309999999999998e-06 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984389+0000", 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1505000000000001e-05 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984396+0000", 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984454+0000", 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8480999999999998e-05 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984556+0000", 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010139 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:49:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.984673+0000", 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117667 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985668+0000", 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099459799999999997 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.986061+0000", 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000392803 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.986103+0000", 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1943999999999997e-05 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.986134+0000", 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0929999999999997e-05 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:622 3.5 e28/22 3:a61279f7:::rbd_data.1179d7437ec6.000000000000001e:head v 28'23, mlcod=28'20)", 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.985111+0000", 2024-04-23T23:49:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.87799590900000002, 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00181605, 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985111+0000", 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985111+0000", 2024-04-23T23:49:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985112+0000", 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985117+0000", 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5210000000000002e-06 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985128+0000", 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985135+0000", 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.007e-06 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985190+0000", 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4589000000000002e-05 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.985209+0000", 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8768999999999999e-05 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.986894+0000", 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016849429999999999 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.986927+0000", 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3473000000000003e-05 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:641 3.18 3:1cb066d0:::rbd_data.1179d7437ec6.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.993835+0000", 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.86927199700000002, 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013444220000000001, 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 641 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993835+0000", 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993835+0000", 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993836+0000", 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6199999999999996e-07 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993840+0000", 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.313e-06 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993851+0000", 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993856+0000", 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2349999999999999e-06 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.993903+0000", 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6916999999999998e-05 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994006+0000", 2024-04-23T23:49:57.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103218 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994133+0000", 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012633699999999999 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994913+0000", 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078032399999999999 2024-04-23T23:49:57.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.995104+0000", 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000190719 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.995141+0000", 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6888999999999999e-05 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.995180+0000", 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8858e-05 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:639 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'32, mlcod=28'28)", 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.994072+0000", 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.86903494400000003, 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017837879999999999, 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994072+0000", 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994072+0000", 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994073+0000", 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-07 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994077+0000", 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6469999999999997e-06 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994088+0000", 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0757999999999999e-05 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994094+0000", 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9499999999999998e-06 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994147+0000", 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-05 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994161+0000", 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3862e-05 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.995819+0000", 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001658416 2024-04-23T23:49:57.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.995856+0000", 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6619000000000003e-05 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:640 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'33, mlcod=28'28)", 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.994827+0000", 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.86828019400000001, 2024-04-23T23:49:57.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001267617, 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994827+0000", 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994827+0000", 2024-04-23T23:49:57.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994828+0000", 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1099999999999995e-07 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994833+0000", 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994846+0000", 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3354e-05 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994852+0000", 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994896+0000", 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4234999999999999e-05 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.994911+0000", 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4871999999999999e-05 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.996076+0000", 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001164537 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.996095+0000", 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8858000000000001e-05 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:676 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'39, mlcod=28'37)", 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.043326+0000", 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.81978108999999999, 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002222258, 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043326+0000", 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043326+0000", 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043327+0000", 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2020000000000001e-06 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043335+0000", 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5349999999999999e-06 2024-04-23T23:49:57.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043348+0000", 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3672000000000001e-05 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043355+0000", 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7039999999999997e-06 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043400+0000", 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5018000000000002e-05 2024-04-23T23:49:57.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.043415+0000", 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5327000000000001e-05 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.045516+0000", 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002100392 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.045548+0000", 2024-04-23T23:49:57.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2407999999999999e-05 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:687 3.c e28/22 3:35136f61:::rbd_object_map.1179d7437ec6:head v 28'41, mlcod=28'39)", 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.070727+0000", 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.79238025099999998, 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087872299999999998, 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070727+0000", 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070727+0000", 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070728+0000", 2024-04-23T23:49:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5489999999999999e-06 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070739+0000", 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0665999999999999e-05 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070758+0000", 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8865999999999999e-05 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070771+0000", 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2903e-05 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070839+0000", 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8332999999999998e-05 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.070858+0000", 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8581e-05 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.071569+0000", 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071144600000000002 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.071606+0000", 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6378999999999998e-05 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:751 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.384396+0000", 2024-04-23T23:49:57.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.478710631, 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012193169999999999, 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 751 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384396+0000", 2024-04-23T23:49:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384396+0000", 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384398+0000", 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.158e-06 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384404+0000", 2024-04-23T23:49:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9009999999999999e-06 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384415+0000", 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1046999999999999e-05 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384421+0000", 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8390000000000001e-06 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:49:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384468+0000", 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6362000000000001e-05 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384561+0000", 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3565000000000003e-05 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.384688+0000", 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126465 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.385325+0000", 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00063758499999999997 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.385553+0000", 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022777499999999999 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.385585+0000", 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1331999999999997e-05 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:49:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.385616+0000", 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1288000000000003e-05 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:49:57.948 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:49:57.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:81 punch from 0xc0b14e1 to 0xc0bcc61, (0xb780 bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:82 read 0x129dcff thru 0x12ad11a (0xf41c bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:83 writesame 0xe918358 thru 0xe926534 data_size 0x1(0xe1dd bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:89 read 0x1ae17ba thru 0x1af02b0 (0xeaf7 bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:90 trunc from 0xe926535 to 0x882feae 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:91 read 0x7bd8ab1 thru 0x7be18c6 (0x8e16 bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:94 read 0x421855e thru 0x4222aa0 (0xa543 bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:97 writesame 0x7755bd1 thru 0x7756711 data_size 0x1(0xb41 bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:98 read 0x404f447 thru 0x405e370 (0xef2a bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:99 writesame 0x901223e thru 0x90131cf data_size 0x1(0xf92 bytes) 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:102 write 0x58d5929 thru 0x58d9dbf (0x4497 bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:103 write 0x5e56a9c thru 0x5e61bb1 (0xb116 bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:104 write 0x52914e7 thru 0x5293add (0x25f7 bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:105 read 0x5ccc2e7 thru 0x5cd630f (0xa029 bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:106 write 0x3399066 thru 0x33a0ded (0x7d88 bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:107 write 0x87eb812 thru 0x87edd83 (0x2572 bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:108 punch from 0x7a37693 to 0x7a444b2, (0xce1f bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:109 read 0x2c900de thru 0x2c9bab9 (0xb9dc bytes) 2024-04-23T23:49:57.953 INFO:teuthology.orchestra.run.smithi092.stdout:110 read 0x7918f33 thru 0x791a987 (0x1a55 bytes) 2024-04-23T23:49:57.953 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:57.963+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:57.954 INFO:teuthology.orchestra.run.smithi092.stdout:112 writesame 0x9b35474 thru 0x9b4431c data_size 0x1(0xeea9 bytes) 2024-04-23T23:49:57.954 INFO:teuthology.orchestra.run.smithi092.stdout:113 writesame 0x1e8a26 thru 0x1f7f08 data_size 0x1(0xf4e3 bytes) 2024-04-23T23:49:57.995 INFO:teuthology.orchestra.run.smithi092.stdout:118 punch from 0xcf4687 to 0xcf9aca, (0x5443 bytes) 2024-04-23T23:49:57.995 INFO:teuthology.orchestra.run.smithi092.stdout:120 read 0x4ab3967 thru 0x4ab6eee (0x3588 bytes) 2024-04-23T23:49:57.998 INFO:teuthology.orchestra.run.smithi092.stdout:121 trunc from 0x9b4431d to 0x763b485 2024-04-23T23:49:58.010 INFO:teuthology.orchestra.run.smithi092.stdout:122 read 0x388740a thru 0x3888af4 (0x16eb bytes) 2024-04-23T23:49:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:49:58.011 INFO:teuthology.orchestra.run.smithi092.stdout:123 read 0x2933788 thru 0x294056f (0xcde8 bytes) 2024-04-23T23:49:58.013 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:49:58.014 INFO:teuthology.orchestra.run.smithi092.stdout:124 write 0x2115bbe thru 0x211c5b3 (0x69f6 bytes) 2024-04-23T23:49:58.014 INFO:teuthology.orchestra.run.smithi092.stdout:126 punch from 0xa21eb0 to 0xa27017, (0x5167 bytes) 2024-04-23T23:49:58.014 INFO:teuthology.orchestra.run.smithi092.stdout:127 read 0xdecefb thru 0xdf106c (0x4172 bytes) 2024-04-23T23:49:58.014 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.025 INFO:teuthology.orchestra.run.smithi092.stdout:129 write 0xd26f891 thru 0xd27878a (0x8efa bytes) 2024-04-23T23:49:58.028 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-23T23:49:57.359955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1209060,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":249,"num_write":205,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219376,"kb_used_data":3188,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755344,"statfs":{"total":773094113280,"available":772869472256,"internally_reserved":0,"allocated":3264512,"data_stored":1934036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":618692,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":200,"num_read_kb":149,"num_write":46,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000563"},"pg_stats":[{"pgid":"3.1f","version":"26'3","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.534751+0000","last_change":"2024-04-23T23:49:53.323402+0000","last_active":"2024-04-23T23:49:55.534751+0000","last_peered":"2024-04-23T23:49:55.534751+0000","last_clean":"2024-04-23T23:49:55.534751+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:49:55.534751+0000","last_undegraded":"2024-04-23T23:49:55.534751+0000","last_fullsized":"2024-04-23T23:49:55.534751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:27.709138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085782299999999996,"stat_sum":{"num_bytes":618608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":3,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337848+0000","last_change":"2024-04-23T23:49:53.334943+0000","last_active":"2024-04-23T23:49:55.337848+0000","last_peered":"2024-04-23T23:49:55.337848+0000","last_clean":"2024-04-23T23:49:55.337848+0000","last_became_active":"2024-04-23T23:49:51.319750+0000","last_became_peered":"2024-04-23T23:49:51.319750+0000","last_unstale":"2024-04-23T23:49:55.337848+0000","last_undegraded":"2024-04-23T23:49:55.337848+0000","last_fullsized":"2024-04-23T23:49:55.337848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:29.453087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067565599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.328510+0000","last_change":"2024-04-23T23:49:53.328510+0000","last_active":"2024-04-23T23:49:53.328510+0000","last_peered":"2024-04-23T23:49:53.328510+0000","last_clean":"2024-04-23T23:49:53.328510+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:49:53.328510+0000","last_undegraded":"2024-04-23T23:49:53.328510+0000","last_fullsized":"2024-04-23T23:49:53.328510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:30:50.310984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018779,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"25'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.329002+0000","last_change":"2024-04-23T23:49:53.328456+0000","last_active":"2024-04-23T23:49:53.329002+0000","last_peered":"2024-04-23T23:49:53.329002+0000","last_clean":"2024-04-23T23:49:53.329002+0000","last_became_active":"2024-04-23T23:49:51.317767+0000","last_became_peered":"2024-04-23T23:49:51.317767+0000","last_unstale":"2024-04-23T23:49:53.329002+0000","last_undegraded":"2024-04-23T23:49:53.329002+0000","last_fullsized":"2024-04-23T23:49:53.329002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:30.222812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099769099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"27'7","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.541051+0000","last_change":"2024-04-23T23:49:53.322810+0000","last_active":"2024-04-23T23:49:55.541051+0000","last_peered":"2024-04-23T23:49:55.541051+0000","last_clean":"2024-04-23T23:49:55.541051+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:49:55.541051+0000","last_undegraded":"2024-04-23T23:49:55.541051+0000","last_fullsized":"2024-04-23T23:49:55.541051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:16:39.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079418399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":53,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"25'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324072+0000","last_change":"2024-04-23T23:49:53.322527+0000","last_active":"2024-04-23T23:49:54.324072+0000","last_peered":"2024-04-23T23:49:54.324072+0000","last_clean":"2024-04-23T23:49:54.324072+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:49:54.324072+0000","last_undegraded":"2024-04-23T23:49:54.324072+0000","last_fullsized":"2024-04-23T23:49:54.324072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:39.253436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041081400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468788+0000","last_change":"2024-04-23T23:49:53.327322+0000","last_active":"2024-04-23T23:49:55.468788+0000","last_peered":"2024-04-23T23:49:55.468788+0000","last_clean":"2024-04-23T23:49:55.468788+0000","last_became_active":"2024-04-23T23:49:51.321539+0000","last_became_peered":"2024-04-23T23:49:51.321539+0000","last_unstale":"2024-04-23T23:49:55.468788+0000","last_undegraded":"2024-04-23T23:49:55.468788+0000","last_fullsized":"2024-04-23T23:49:55.468788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:15.545560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038598300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"26'2","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337724+0000","last_change":"2024-04-23T23:49:53.333734+0000","last_active":"2024-04-23T23:49:55.337724+0000","last_peered":"2024-04-23T23:49:55.337724+0000","last_clean":"2024-04-23T23:49:55.337724+0000","last_became_active":"2024-04-23T23:49:51.321251+0000","last_became_peered":"2024-04-23T23:49:51.321251+0000","last_unstale":"2024-04-23T23:49:55.337724+0000","last_undegraded":"2024-04-23T23:49:55.337724+0000","last_fullsized":"2024-04-23T23:49:55.337724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:13.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064860299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328796+0000","last_change":"2024-04-23T23:49:53.323222+0000","last_active":"2024-04-23T23:49:55.328796+0000","last_peered":"2024-04-23T23:49:55.328796+0000","last_clean":"2024-04-23T23:49:55.328796+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:49:55.328796+0000","last_undegraded":"2024-04-23T23:49:55.328796+0000","last_fullsized":"2024-04-23T23:49:55.328796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:26:05.273173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091814399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.327873+0000","last_change":"2024-04-23T23:49:53.327873+0000","last_active":"2024-04-23T23:49:53.327873+0000","last_peered":"2024-04-23T23:49:53.327873+0000","last_clean":"2024-04-23T23:49:53.327873+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:49:53.327873+0000","last_undegraded":"2024-04-23T23:49:53.327873+0000","last_fullsized":"2024-04-23T23:49:53.327873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:12:57.310031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039462200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328091+0000","last_change":"2024-04-23T23:49:53.781187+0000","last_active":"2024-04-23T23:49:54.328091+0000","last_peered":"2024-04-23T23:49:54.328091+0000","last_clean":"2024-04-23T23:49:54.328091+0000","last_became_active":"2024-04-23T23:49:51.321623+0000","last_became_peered":"2024-04-23T23:49:51.321623+0000","last_unstale":"2024-04-23T23:49:54.328091+0000","last_undegraded":"2024-04-23T23:49:54.328091+0000","last_fullsized":"2024-04-23T23:49:54.328091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:47:06.585826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066676300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324344+0000","last_change":"2024-04-23T23:49:53.322838+0000","last_active":"2024-04-23T23:49:54.324344+0000","last_peered":"2024-04-23T23:49:54.324344+0000","last_clean":"2024-04-23T23:49:54.324344+0000","last_became_active":"2024-04-23T23:49:51.320365+0000","last_became_peered":"2024-04-23T23:49:51.320365+0000","last_unstale":"2024-04-23T23:49:54.324344+0000","last_undegraded":"2024-04-23T23:49:54.324344+0000","last_fullsized":"2024-04-23T23:49:54.324344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:04.801819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042792500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.335655+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:49:54.335655+0000","last_peered":"2024-04-23T23:49:54.335655+0000","last_clean":"2024-04-23T23:49:54.335655+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:49:54.335655+0000","last_undegraded":"2024-04-23T23:49:54.335655+0000","last_fullsized":"2024-04-23T23:49:54.335655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328797+0000","last_change":"2024-04-23T23:49:53.322476+0000","last_active":"2024-04-23T23:49:55.328797+0000","last_peered":"2024-04-23T23:49:55.328797+0000","last_clean":"2024-04-23T23:49:55.328797+0000","last_became_active":"2024-04-23T23:49:51.317534+0000","last_became_peered":"2024-04-23T23:49:51.317534+0000","last_unstale":"2024-04-23T23:49:55.328797+0000","last_undegraded":"2024-04-23T23:49:55.328797+0000","last_fullsized":"2024-04-23T23:49:55.328797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:10.088753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046868700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468960+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:49:55.468960+0000","last_peered":"2024-04-23T23:49:55.468960+0000","last_clean":"2024-04-23T23:49:55.468960+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:49:55.468960+0000","last_undegraded":"2024-04-23T23:49:55.468960+0000","last_fullsized":"2024-04-23T23:49:55.468960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337896+0000","last_change":"2024-04-23T23:49:53.334802+0000","last_active":"2024-04-23T23:49:55.337896+0000","last_peered":"2024-04-23T23:49:55.337896+0000","last_clean":"2024-04-23T23:49:55.337896+0000","last_became_active":"2024-04-23T23:49:51.320498+0000","last_became_peered":"2024-04-23T23:49:51.320498+0000","last_unstale":"2024-04-23T23:49:55.337896+0000","last_undegraded":"2024-04-23T23:49:55.337896+0000","last_fullsized":"2024-04-23T23:49:55.337896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:04:31.101139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010602859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328187+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:49:54.328187+0000","last_peered":"2024-04-23T23:49:54.328187+0000","last_clean":"2024-04-23T23:49:54.328187+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:49:54.328187+0000","last_undegraded":"2024-04-23T23:49:54.328187+0000","last_fullsized":"2024-04-23T23:49:54.328187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468979+0000","last_change":"2024-04-23T23:49:53.327533+0000","last_active":"2024-04-23T23:49:55.468979+0000","last_peered":"2024-04-23T23:49:55.468979+0000","last_clean":"2024-04-23T23:49:55.468979+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:49:55.468979+0000","last_undegraded":"2024-04-23T23:49:55.468979+0000","last_fullsized":"2024-04-23T23:49:55.468979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:32.879558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056687800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.469105+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:49:55.469105+0000","last_peered":"2024-04-23T23:49:55.469105+0000","last_clean":"2024-04-23T23:49:55.469105+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:49:55.469105+0000","last_undegraded":"2024-04-23T23:49:55.469105+0000","last_fullsized":"2024-04-23T23:49:55.469105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'2","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.345808+0000","last_change":"2024-04-23T23:49:53.328813+0000","last_active":"2024-04-23T23:49:53.345808+0000","last_peered":"2024-04-23T23:49:53.345808+0000","last_clean":"2024-04-23T23:49:53.345808+0000","last_became_active":"2024-04-23T23:49:51.319410+0000","last_became_peered":"2024-04-23T23:49:51.319410+0000","last_unstale":"2024-04-23T23:49:53.345808+0000","last_undegraded":"2024-04-23T23:49:53.345808+0000","last_fullsized":"2024-04-23T23:49:53.345808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:22.816805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154289,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324216+0000","last_change":"2024-04-23T23:49:53.322781+0000","last_active":"2024-04-23T23:49:54.324216+0000","last_peered":"2024-04-23T23:49:54.324216+0000","last_clean":"2024-04-23T23:49:54.324216+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:49:54.324216+0000","last_undegraded":"2024-04-23T23:49:54.324216+0000","last_fullsized":"2024-04-23T23:49:54.324216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:50:09.670335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054288899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.327325+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:49:53.327325+0000","last_peered":"2024-04-23T23:49:53.327325+0000","last_clean":"2024-04-23T23:49:53.327325+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:49:53.327325+0000","last_undegraded":"2024-04-23T23:49:53.327325+0000","last_fullsized":"2024-04-23T23:49:53.327325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:39.497845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.468795+0000","last_change":"2024-04-23T23:49:53.327404+0000","last_active":"2024-04-23T23:49:55.468795+0000","last_peered":"2024-04-23T23:49:55.468795+0000","last_clean":"2024-04-23T23:49:55.468795+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:49:55.468795+0000","last_undegraded":"2024-04-23T23:49:55.468795+0000","last_fullsized":"2024-04-23T23:49:55.468795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:16.224406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043824499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.322309+0000","last_change":"2024-04-23T23:49:43.280465+0000","last_active":"2024-04-23T23:49:53.322309+0000","last_peered":"2024-04-23T23:49:53.322309+0000","last_clean":"2024-04-23T23:49:53.322309+0000","last_became_active":"2024-04-23T23:49:43.280144+0000","last_became_peered":"2024-04-23T23:49:43.280144+0000","last_unstale":"2024-04-23T23:49:53.322309+0000","last_undegraded":"2024-04-23T23:49:53.322309+0000","last_fullsized":"2024-04-23T23:49:53.322309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:42:45.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328608+0000","last_change":"2024-04-23T23:49:53.322700+0000","last_active":"2024-04-23T23:49:55.328608+0000","last_peered":"2024-04-23T23:49:55.328608+0000","last_clean":"2024-04-23T23:49:55.328608+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:49:55.328608+0000","last_undegraded":"2024-04-23T23:49:55.328608+0000","last_fullsized":"2024-04-23T23:49:55.328608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:58.702424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085923399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328150+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:49:54.328150+0000","last_peered":"2024-04-23T23:49:54.328150+0000","last_clean":"2024-04-23T23:49:54.328150+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:49:54.328150+0000","last_undegraded":"2024-04-23T23:49:54.328150+0000","last_fullsized":"2024-04-23T23:49:54.328150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:28:10.960350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"27'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.517355+0000","last_change":"2024-04-23T23:49:53.334574+0000","last_active":"2024-04-23T23:49:55.517355+0000","last_peered":"2024-04-23T23:49:55.517355+0000","last_clean":"2024-04-23T23:49:55.517355+0000","last_became_active":"2024-04-23T23:49:51.318301+0000","last_became_peered":"2024-04-23T23:49:51.318301+0000","last_unstale":"2024-04-23T23:49:55.517355+0000","last_undegraded":"2024-04-23T23:49:55.517355+0000","last_fullsized":"2024-04-23T23:49:55.517355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:19.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011802519999999999,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.327317+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:49:53.327317+0000","last_peered":"2024-04-23T23:49:53.327317+0000","last_clean":"2024-04-23T23:49:53.327317+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:49:53.327317+0000","last_undegraded":"2024-04-23T23:49:53.327317+0000","last_fullsized":"2024-04-23T23:49:53.327317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328212+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:49:54.328212+0000","last_peered":"2024-04-23T23:49:54.328212+0000","last_clean":"2024-04-23T23:49:54.328212+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:49:54.328212+0000","last_undegraded":"2024-04-23T23:49:54.328212+0000","last_fullsized":"2024-04-23T23:49:54.328212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.337712+0000","last_change":"2024-04-23T23:49:53.333756+0000","last_active":"2024-04-23T23:49:55.337712+0000","last_peered":"2024-04-23T23:49:55.337712+0000","last_clean":"2024-04-23T23:49:55.337712+0000","last_became_active":"2024-04-23T23:49:51.317708+0000","last_became_peered":"2024-04-23T23:49:51.317708+0000","last_unstale":"2024-04-23T23:49:55.337712+0000","last_undegraded":"2024-04-23T23:49:55.337712+0000","last_fullsized":"2024-04-23T23:49:55.337712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:09:35.137542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041896000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.324272+0000","last_change":"2024-04-23T23:49:53.322582+0000","last_active":"2024-04-23T23:49:54.324272+0000","last_peered":"2024-04-23T23:49:54.324272+0000","last_clean":"2024-04-23T23:49:54.324272+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:49:54.324272+0000","last_undegraded":"2024-04-23T23:49:54.324272+0000","last_fullsized":"2024-04-23T23:49:54.324272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:56:44.873026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045422700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.335720+0000","last_change":"2024-04-23T23:49:53.494617+0000","last_active":"2024-04-23T23:49:54.335720+0000","last_peered":"2024-04-23T23:49:54.335720+0000","last_clean":"2024-04-23T23:49:54.335720+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:49:54.335720+0000","last_undegraded":"2024-04-23T23:49:54.335720+0000","last_fullsized":"2024-04-23T23:49:54.335720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:39.495902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039630999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"27'22","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.340620+0000","last_change":"2024-04-23T23:49:53.334695+0000","last_active":"2024-04-23T23:49:55.340620+0000","last_peered":"2024-04-23T23:49:55.340620+0000","last_clean":"2024-04-23T23:49:55.340620+0000","last_became_active":"2024-04-23T23:49:51.317643+0000","last_became_peered":"2024-04-23T23:49:51.317643+0000","last_unstale":"2024-04-23T23:49:55.340620+0000","last_undegraded":"2024-04-23T23:49:55.340620+0000","last_fullsized":"2024-04-23T23:49:55.340620+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:26.033458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001287243,"stat_sum":{"num_bytes":49,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":90,"num_read_kb":68,"num_write":31,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T23:49:53.328808+0000","last_change":"2024-04-23T23:49:53.328808+0000","last_active":"2024-04-23T23:49:53.328808+0000","last_peered":"2024-04-23T23:49:53.328808+0000","last_clean":"2024-04-23T23:49:53.328808+0000","last_became_active":"2024-04-23T23:49:51.319004+0000","last_became_peered":"2024-04-23T23:49:51.319004+0000","last_unstale":"2024-04-23T23:49:53.328808+0000","last_undegraded":"2024-04-23T23:49:53.328808+0000","last_fullsized":"2024-04-23T23:49:53.328808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:33:26.486284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012607440000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328162+0000","last_change":"2024-04-23T23:49:53.781509+0000","last_active":"2024-04-23T23:49:54.328162+0000","last_peered":"2024-04-23T23:49:54.328162+0000","last_clean":"2024-04-23T23:49:54.328162+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:49:54.328162+0000","last_undegraded":"2024-04-23T23:49:54.328162+0000","last_fullsized":"2024-04-23T23:49:54.328162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:55.322236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090598199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328220+0000","last_change":"2024-04-23T23:49:53.781245+0000","last_active":"2024-04-23T23:49:54.328220+0000","last_peered":"2024-04-23T23:49:54.328220+0000","last_clean":"2024-04-23T23:49:54.328220+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:49:54.328220+0000","last_undegraded":"2024-04-23T23:49:54.328220+0000","last_fullsized":"2024-04-23T23:49:54.328220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:09:58.407504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071364900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328329+0000","last_change":"2024-04-23T23:49:53.781604+0000","last_active":"2024-04-23T23:49:54.328329+0000","last_peered":"2024-04-23T23:49:54.328329+0000","last_clean":"2024-04-23T23:49:54.328329+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:49:54.328329+0000","last_undegraded":"2024-04-23T23:49:54.328329+0000","last_fullsized":"2024-04-23T23:49:54.328329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:01.557911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097850600000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.335740+0000","last_change":"2024-04-23T23:49:53.494621+0000","last_active":"2024-04-23T23:49:54.335740+0000","last_peered":"2024-04-23T23:49:54.335740+0000","last_clean":"2024-04-23T23:49:54.335740+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:49:54.335740+0000","last_undegraded":"2024-04-23T23:49:54.335740+0000","last_fullsized":"2024-04-23T23:49:54.335740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:42.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400901,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328077+0000","last_change":"2024-04-23T23:49:53.780895+0000","last_active":"2024-04-23T23:49:54.328077+0000","last_peered":"2024-04-23T23:49:54.328077+0000","last_clean":"2024-04-23T23:49:54.328077+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:49:54.328077+0000","last_undegraded":"2024-04-23T23:49:54.328077+0000","last_fullsized":"2024-04-23T23:49:54.328077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:17:01.882241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049304299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-23T23:49:55.328575+0000","last_change":"2024-04-23T23:49:53.323023+0000","last_active":"2024-04-23T23:49:55.328575+0000","last_peered":"2024-04-23T23:49:55.328575+0000","last_clean":"2024-04-23T23:49:55.328575+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:49:55.328575+0000","last_undegraded":"2024-04-23T23:49:55.328575+0000","last_fullsized":"2024-04-23T23:49:55.328575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:58.675393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000902432,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-23T23:49:54.328023+0000","last_change":"2024-04-23T23:49:53.780967+0000","last_active":"2024-04-23T23:49:54.328023+0000","last_peered":"2024-04-23T23:49:54.328023+0000","last_clean":"2024-04-23T23:49:54.328023+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:49:54.328023+0000","last_undegraded":"2024-04-23T23:49:54.328023+0000","last_fullsized":"2024-04-23T23:49:54.328023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:43.662867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053415600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":618692,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":200,"num_read_kb":149,"num_write":46,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":100823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":94988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":685383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":685337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":88766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":94985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":94988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":88766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:49:58.031 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T23:49:58.031 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:49:58.031 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T23:49:58.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T23:49:58.032 INFO:teuthology.orchestra.run.smithi092.stdout:131 read 0x3d5fc58 thru 0x3d69f54 (0xa2fd bytes) 2024-04-23T23:49:58.032 INFO:teuthology.orchestra.run.smithi092.stdout:133 punch from 0xa1f9de5 to 0xa1fa921, (0xb3c bytes) 2024-04-23T23:49:58.032 INFO:teuthology.orchestra.run.smithi092.stdout:134 read 0xd1ab5b9 thru 0xd1bad1c (0xf764 bytes) 2024-04-23T23:49:58.032 INFO:teuthology.orchestra.run.smithi092.stdout:135 punch from 0x133f089 to 0x1346be2, (0x7b59 bytes) 2024-04-23T23:49:58.035 INFO:teuthology.orchestra.run.smithi092.stdout:136 writesame 0xda13ddf thru 0xda1b023 data_size 0x31(0x7245 bytes) 2024-04-23T23:49:58.035 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.035 INFO:teuthology.orchestra.run.smithi092.stdout:138 writesame 0x94de093 thru 0x94e2d61 data_size 0x7(0x4ccf bytes) 2024-04-23T23:49:58.038 INFO:teuthology.orchestra.run.smithi092.stdout:139 write 0x5dc3a25 thru 0x5dd1bb5 (0xe191 bytes) 2024-04-23T23:49:58.040 INFO:teuthology.orchestra.run.smithi092.stdout:143 write 0xbb40905 thru 0xbb49299 (0x8995 bytes) 2024-04-23T23:49:58.043 INFO:teuthology.orchestra.run.smithi092.stdout:144 write 0x59bbe0c thru 0x59c2fea (0x71df bytes) 2024-04-23T23:49:58.045 INFO:teuthology.orchestra.run.smithi092.stdout:145 writesame 0x7aa4aed thru 0x7aadedc data_size 0x48(0x93f0 bytes) 2024-04-23T23:49:58.050 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:58.064+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:58.059 INFO:teuthology.orchestra.run.smithi092.stdout:147 trunc from 0xda1b024 to 0x4a26907 2024-04-23T23:49:58.060 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:58.060 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:58.061 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:49:58.061 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:58.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:49:58.090 INFO:teuthology.orchestra.run.smithi092.stdout:148 write 0x889458e thru 0x88a15d6 (0xd049 bytes) 2024-04-23T23:49:58.093 INFO:teuthology.orchestra.run.smithi092.stdout:153 trunc from 0x88a15d7 to 0xa641c23 2024-04-23T23:49:58.101 INFO:teuthology.orchestra.run.smithi092.stdout:154 punch from 0x1d16fbf to 0x1d22e4a, (0xbe8b bytes) 2024-04-23T23:49:58.101 INFO:teuthology.orchestra.run.smithi092.stdout:155 write 0x98e457b thru 0x98ed478 (0x8efe bytes) 2024-04-23T23:49:58.105 INFO:teuthology.orchestra.run.smithi092.stdout:156 write 0x648630a thru 0x6491b94 (0xb88b bytes) 2024-04-23T23:49:58.106 INFO:teuthology.orchestra.run.smithi092.stdout:157 read 0x32cc815 thru 0x32ce2cf (0x1abb bytes) 2024-04-23T23:49:58.107 INFO:teuthology.orchestra.run.smithi092.stdout:158 writesame 0x26d9e0d thru 0x26e9574 data_size 0xa8(0xf768 bytes) 2024-04-23T23:49:58.115 INFO:teuthology.orchestra.run.smithi092.stdout:159 write 0xbb2c120 thru 0xbb325f2 (0x64d3 bytes) 2024-04-23T23:49:58.118 INFO:teuthology.orchestra.run.smithi092.stdout:160 write 0x897ecdc thru 0x89863f4 (0x7719 bytes) 2024-04-23T23:49:58.119 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.119 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.119 INFO:teuthology.orchestra.run.smithi092.stdout:165 write 0x60a3e8d thru 0x60a5cd8 (0x1e4c bytes) 2024-04-23T23:49:58.121 INFO:teuthology.orchestra.run.smithi092.stdout:166 read 0x5fd107d thru 0x5fd4d35 (0x3cb9 bytes) 2024-04-23T23:49:58.121 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.121 INFO:teuthology.orchestra.run.smithi092.stdout:168 write 0x6668496 thru 0x66705ec (0x8157 bytes) 2024-04-23T23:49:58.132 INFO:teuthology.orchestra.run.smithi092.stdout:169 trunc from 0xbb325f3 to 0x3c9af8a 2024-04-23T23:49:58.143 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe628d42 2024-04-23T23:49:58.151 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:49:58.165+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:49:58.155 INFO:teuthology.orchestra.run.smithi092.stdout:171 trunc from 0x3c9af8a to 0xe628d42 2024-04-23T23:49:58.162 INFO:teuthology.orchestra.run.smithi092.stdout:172 read 0x2e8abde thru 0x2e8ead2 (0x3ef5 bytes) 2024-04-23T23:49:58.162 INFO:teuthology.orchestra.run.smithi092.stdout:173 writesame 0x3379256 thru 0x338669b data_size 0x9(0xd446 bytes) 2024-04-23T23:49:58.171 INFO:teuthology.orchestra.run.smithi092.stdout:174 writesame 0xea27f0a thru 0xea291b4 data_size 0x1b(0x12ab bytes) 2024-04-23T23:49:58.171 INFO:teuthology.orchestra.run.smithi092.stdout:175 writesame 0x8e5f714 thru 0x8e60751 data_size 0x21(0x103e bytes) 2024-04-23T23:49:58.173 INFO:teuthology.orchestra.run.smithi092.stdout:176 read 0xdd654bb thru 0xdd69260 (0x3da6 bytes) 2024-04-23T23:49:58.173 INFO:teuthology.orchestra.run.smithi092.stdout:177 write 0xd34d933 thru 0xd34ea8c (0x115a bytes) 2024-04-23T23:49:58.176 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.176 INFO:teuthology.orchestra.run.smithi092.stdout:179 read 0xe49a11e thru 0xe4a0abc (0x699f bytes) 2024-04-23T23:49:58.176 INFO:teuthology.orchestra.run.smithi092.stdout:180 punch from 0x90accea to 0x90ae435, (0x174b bytes) 2024-04-23T23:49:58.176 INFO:teuthology.orchestra.run.smithi092.stdout:181 write 0x6d32f21 thru 0x6d357e4 (0x28c4 bytes) 2024-04-23T23:49:58.177 INFO:teuthology.orchestra.run.smithi092.stdout:183 read 0x83e1bd3 thru 0x83e70f1 (0x551f bytes) 2024-04-23T23:49:58.177 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:185 punch from 0xce8daee to 0xce9b4ca, (0xd9dc bytes) 2024-04-23T23:49:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:186 write 0xbcd1ccb thru 0xbcd4add (0x2e13 bytes) 2024-04-23T23:49:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:188 read 0x1407b1 thru 0x14662e (0x5e7e bytes) 2024-04-23T23:49:58.179 INFO:teuthology.orchestra.run.smithi092.stdout:189 read 0xd9ff01c thru 0xda055b0 (0x6595 bytes) 2024-04-23T23:49:58.179 INFO:teuthology.orchestra.run.smithi092.stdout:190 flatten 2024-04-23T23:49:58.184 INFO:teuthology.orchestra.run.smithi092.stdout:191 write 0x431c7ba thru 0x43223db (0x5c22 bytes) 2024-04-23T23:49:58.186 INFO:teuthology.orchestra.run.smithi092.stdout:192 read 0x55e8680 thru 0x55f7e35 (0xf7b6 bytes) 2024-04-23T23:49:58.186 INFO:teuthology.orchestra.run.smithi092.stdout:193 write 0x24fbfac thru 0x2502af9 (0x6b4e bytes) 2024-04-23T23:49:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:58.194 INFO:teuthology.orchestra.run.smithi092.stdout:194 trunc from 0xea291b5 to 0x9d92aa9 2024-04-23T23:49:58.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:58.194 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:58.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T23:49:58.211 INFO:teuthology.orchestra.run.smithi092.stdout:195 write 0xd2a533f thru 0xd2adb75 (0x8837 bytes) 2024-04-23T23:49:58.215 INFO:teuthology.orchestra.run.smithi092.stdout:196 read 0xcef5d23 thru 0xcef7429 (0x1707 bytes) 2024-04-23T23:49:58.215 INFO:teuthology.orchestra.run.smithi092.stdout:197 read 0x2591520 thru 0x2596785 (0x5266 bytes) 2024-04-23T23:49:58.216 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:49:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:49:58.217 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:58.223 INFO:teuthology.orchestra.run.smithi092.stdout:198 write 0xe78601a thru 0xe78d3f1 (0x73d8 bytes) 2024-04-23T23:49:58.224 INFO:teuthology.orchestra.run.smithi092.stdout:199 read 0x9f019ad thru 0x9f0b41e (0x9a72 bytes) 2024-04-23T23:49:58.224 INFO:teuthology.orchestra.run.smithi092.stdout:200 read 0x336da6e thru 0x3378b4a (0xb0dd bytes) 2024-04-23T23:49:58.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:49:58.229 INFO:teuthology.orchestra.run.smithi092.stdout:201 trunc from 0xe78d3f2 to 0xb1e8311 2024-04-23T23:49:58.236 INFO:teuthology.orchestra.run.smithi092.stdout:204 punch from 0xaee856c to 0xaeedc1f, (0x56b3 bytes) 2024-04-23T23:49:58.236 INFO:teuthology.orchestra.run.smithi092.stdout:205 write 0xf548b4 thru 0xf60d50 (0xc49d bytes) 2024-04-23T23:49:58.237 INFO:teuthology.orchestra.run.smithi092.stdout:206 writesame 0x9bf0451 thru 0x9bfea70 data_size 0x38(0xe620 bytes) 2024-04-23T23:49:58.241 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.250 INFO:teuthology.orchestra.run.smithi092.stdout:208 write 0xe854500 thru 0xe854906 (0x407 bytes) 2024-04-23T23:49:58.251 INFO:teuthology.orchestra.run.smithi092.stdout:209 write 0x9730f9 thru 0x97811b (0x5023 bytes) 2024-04-23T23:49:58.252 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:58.265+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:58.253 INFO:teuthology.orchestra.run.smithi092.stdout:210 write 0x654ff7c thru 0x655f50a (0xf58f bytes) 2024-04-23T23:49:58.255 INFO:teuthology.orchestra.run.smithi092.stdout:211 write 0x237fc13 thru 0x2388de7 (0x91d5 bytes) 2024-04-23T23:49:58.257 INFO:teuthology.orchestra.run.smithi092.stdout:213 punch from 0xe82c1e5 to 0xe836326, (0xa141 bytes) 2024-04-23T23:49:58.270 INFO:teuthology.orchestra.run.smithi092.stdout:214 trunc from 0xe854907 to 0x38458e1 2024-04-23T23:49:58.294 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.294 INFO:teuthology.orchestra.run.smithi092.stdout:216 write 0x34e04a thru 0x353080 (0x5037 bytes) 2024-04-23T23:49:58.295 INFO:teuthology.orchestra.run.smithi092.stdout:217 write 0x24ff242 thru 0x250e6ee (0xf4ad bytes) 2024-04-23T23:49:58.297 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.297 INFO:teuthology.orchestra.run.smithi092.stdout:219 read 0xd44185 thru 0xd45e1d (0x1c99 bytes) 2024-04-23T23:49:58.297 INFO:teuthology.orchestra.run.smithi092.stdout:220 read 0x3213fe8 thru 0x3216121 (0x213a bytes) 2024-04-23T23:49:58.298 INFO:teuthology.orchestra.run.smithi092.stdout:221 punch from 0xec0567 to 0xec4178, (0x3c11 bytes) 2024-04-23T23:49:58.298 INFO:teuthology.orchestra.run.smithi092.stdout:222 read 0x2df10d3 thru 0x2dff161 (0xe08f bytes) 2024-04-23T23:49:58.310 INFO:teuthology.orchestra.run.smithi092.stdout:223 writesame 0x94f178c thru 0x94f3d56 data_size 0x19(0x25cb bytes) 2024-04-23T23:49:58.310 INFO:teuthology.orchestra.run.smithi092.stdout:225 write 0xcca5c4 thru 0xcce0f5 (0x3b32 bytes) 2024-04-23T23:49:58.311 INFO:teuthology.orchestra.run.smithi092.stdout:226 read 0x2f81197 thru 0x2f8148d (0x2f7 bytes) 2024-04-23T23:49:58.312 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.317 INFO:teuthology.orchestra.run.smithi092.stdout:228 trunc from 0x94f3d57 to 0xe00e91f 2024-04-23T23:49:58.323 INFO:teuthology.orchestra.run.smithi092.stdout:229 trunc from 0xe00e91f to 0xe5fcbf9 2024-04-23T23:49:58.329 INFO:teuthology.orchestra.run.smithi092.stdout:230 punch from 0x83acead to 0x83b90f5, (0xc248 bytes) 2024-04-23T23:49:58.329 INFO:teuthology.orchestra.run.smithi092.stdout:231 read 0xe390ad5 thru 0xe39a87b (0x9da7 bytes) 2024-04-23T23:49:58.329 INFO:teuthology.orchestra.run.smithi092.stdout:232 write 0x34769a2 thru 0x347aa7f (0x40de bytes) 2024-04-23T23:49:58.330 INFO:teuthology.orchestra.run.smithi092.stdout:233 read 0x7d4d2ed thru 0x7d5757c (0xa290 bytes) 2024-04-23T23:49:58.331 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.332 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:58.332 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:49:58.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:49:58.332 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:49:58.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:52.319026+0000", 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.026063647, 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00159435, 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.333 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319026+0000", 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319026+0000", 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319027+0000", 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3510000000000001e-06 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319036+0000", 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9439999999999994e-06 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319049+0000", 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2670000000000001e-05 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319056+0000", 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5900000000000002e-06 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319107+0000", 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0812000000000001e-05 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319190+0000", 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3265999999999999e-05 2024-04-23T23:49:58.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319283+0000", 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2601000000000003e-05 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.319905+0000", 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062184600000000001 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.320558+0000", 2024-04-23T23:49:58.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000653452 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.320590+0000", 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1968000000000003e-05 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:52.320620+0000", 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9850000000000001e-05 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:53.323663+0000", 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0214260629999998, 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001488767, 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 9 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.339 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323663+0000", 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323663+0000", 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323665+0000", 2024-04-23T23:49:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1820000000000001e-06 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323671+0000", 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1489999999999999e-06 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323682+0000", 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1551e-05 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323692+0000", 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.876e-06 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323740+0000", 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7648000000000002e-05 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323806+0000", 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6589999999999998e-05 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.323909+0000", 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102367 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.324549+0000", 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00064017299999999996 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.325089+0000", 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00053971800000000001 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.325123+0000", 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4309e-05 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:53.325152+0000", 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9204e-05 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:595 3.14 3:2df57bc2:::rbd_data.1179d7437ec6.0000000000000010:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 765729~33821] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.939089+0000", 2024-04-23T23:49:58.344 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.4060000129999999, 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014603999999999999, 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 595 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939089+0000", 2024-04-23T23:49:58.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939089+0000", 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939090+0000", 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1500000000000004e-07 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939165+0000", 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4598000000000006e-05 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939175+0000", 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0564000000000001e-05 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939180+0000", 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.006e-06 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939219+0000", 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8612999999999998e-05 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939298+0000", 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9086999999999997e-05 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.939424+0000", 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000126196 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.940136+0000", 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00071135299999999996 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.940493+0000", 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035791600000000001 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.940525+0000", 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1402999999999997e-05 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.940550+0000", 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4949000000000001e-05 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:620 3.1b e28/22 3:dc04aecc:::rbd_data.1179d7437ec6.000000000000001c:head v 28'14, mlcod=28'12)", 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.984778+0000", 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.360311625, 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001526157, 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.350 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984778+0000", 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984778+0000", 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984778+0000", 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5700000000000002e-07 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984784+0000", 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7760000000000003e-06 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984795+0000", 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0864e-05 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984803+0000", 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8229999999999995e-06 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984892+0000", 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9201000000000002e-05 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.984912+0000", 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9457999999999999e-05 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.986268+0000", 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013567379999999999 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.986304+0000", 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5639999999999998e-05 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:624 3.1c e28/22 3:3d167183:::rbd_data.1179d7437ec6.0000000000000020:head v 28'6, mlcod=28'4)", 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.985082+0000", 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.360007134, 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038804550000000001, 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985082+0000", 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985082+0000", 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985083+0000", 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5499999999999997e-07 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985089+0000", 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0159999999999997e-06 2024-04-23T23:49:58.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985099+0000", 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0071000000000001e-05 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985106+0000", 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7059999999999998e-06 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985151+0000", 2024-04-23T23:49:58.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5305000000000001e-05 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985169+0000", 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8119999999999999e-05 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.988933+0000", 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037637830000000001 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.988963+0000", 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9598999999999999e-05 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:749 3.3 3:c01e78db:::rbd_header.1179984961fa:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.381565+0000", 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.96352470400000001, 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001435787, 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 749 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381565+0000", 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381565+0000", 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381565+0000", 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0200000000000001e-07 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381570+0000", 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2640000000000002e-06 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381580+0000", 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9680000000000001e-06 2024-04-23T23:49:58.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381585+0000", 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4020000000000001e-06 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381635+0000", 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9536999999999997e-05 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381754+0000", 2024-04-23T23:49:58.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119364 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.381982+0000", 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022772100000000001 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.382484+0000", 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000502547 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.382931+0000", 2024-04-23T23:49:58.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044717800000000001 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.382964+0000", 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2870000000000002e-05 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.383000+0000", 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6233999999999999e-05 2024-04-23T23:49:58.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:959 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.883562+0000", 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.46152707199999998, 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014018819999999999, 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 959 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883562+0000", 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883562+0000", 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883563+0000", 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4600000000000005e-07 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883566+0000", 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.824e-06 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883573+0000", 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6699999999999994e-06 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883579+0000", 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.02e-06 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883631+0000", 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1881e-05 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883675+0000", 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3831000000000001e-05 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.883791+0000", 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011560100000000001 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.884265+0000", 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047382000000000001 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.884905+0000", 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00064033899999999997 2024-04-23T23:49:58.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.884939+0000", 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4121000000000001e-05 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.884964+0000", 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5228999999999998e-05 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.372 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1143 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.42419352999999999, 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017226057999999999, 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1143 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920898+0000", 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.846e-06 2024-04-23T23:49:58.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923748+0000", 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028504649999999999 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923773+0000", 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4311000000000002e-05 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923784+0000", 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1212e-05 2024-04-23T23:49:58.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923853+0000", 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9572000000000004e-05 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923965+0000", 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001114 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924349+0000", 2024-04-23T23:49:58.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038400400000000002 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937275+0000", 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012926725 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938026+0000", 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075019300000000002 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938068+0000", 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2809999999999998e-05 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938122+0000", 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3520000000000003e-05 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1142 3.1b e29/22 3:dd409f93:::rbd_data.1179984961fa.0000000000000002:head v 29'17, mlcod=29'15)", 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:49:58.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.42102606300000001, 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014083770000000001, 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:49:58.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924064+0000", 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7000000000000003e-07 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929770+0000", 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057057829999999999 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929783+0000", 2024-04-23T23:49:58.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3118999999999999e-05 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929792+0000", 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6549999999999993e-06 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929854+0000", 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2498999999999999e-05 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929871+0000", 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7007000000000002e-05 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938109+0000", 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082374490000000009 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938147+0000", 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8287999999999997e-05 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1148 3.a e29/22 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head v 29'8, mlcod=28'6)", 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41470722199999999, 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067411720000000001, 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:49:58.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930383+0000", 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8999999999999995e-07 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930391+0000", 2024-04-23T23:49:58.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7209999999999994e-06 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930402+0000", 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0986e-05 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930408+0000", 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.703e-06 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930466+0000", 2024-04-23T23:49:58.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7485000000000003e-05 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930485+0000", 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8936e-05 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937077+0000", 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065921360000000002 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937123+0000", 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6415000000000002e-05 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1194 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [stat,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.017188+0000", 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.32790180200000002, 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015628700000000001, 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1194 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017188+0000", 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017188+0000", 2024-04-23T23:49:58.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017190+0000", 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.018e-06 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017197+0000", 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0709999999999997e-06 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017212+0000", 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5316999999999999e-05 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017222+0000", 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8300000000000008e-06 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017302+0000", 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0098999999999993e-05 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017396+0000", 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4170000000000001e-05 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.017537+0000", 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140851 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.018376+0000", 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00083920100000000003 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.018686+0000", 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309394 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.018721+0000", 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5837999999999998e-05 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.018750+0000", 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9081000000000002e-05 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.392 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1200 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.019050+0000", 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.32603977699999998, 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026167410000000001, 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.393 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1200 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019050+0000", 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019050+0000", 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019050+0000", 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4e-07 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019054+0000", 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6509999999999999e-06 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019063+0000", 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019069+0000", 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1749999999999996e-06 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019108+0000", 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9344999999999997e-05 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019157+0000", 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400999999999997e-05 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019291+0000", 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013359299999999999 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.019888+0000", 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059743700000000003 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.021612+0000", 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017233050000000001 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.021641+0000", 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9713000000000002e-05 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.021666+0000", 2024-04-23T23:49:58.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5045000000000002e-05 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1236 3.3 3:c019f6e6:::rbd_data.1179984961fa.000000000000000a:head [set-alloc-hint object_size 16777216 write_size 16777216,write 647315~19663] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.051869+0000", 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.293220641, 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013578819999999999, 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1236 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051869+0000", 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051869+0000", 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051870+0000", 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7300000000000005e-07 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051885+0000", 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5417000000000002e-05 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051898+0000", 2024-04-23T23:49:58.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-05 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051903+0000", 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7899999999999996e-06 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.051942+0000", 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8458999999999999e-05 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.052024+0000", 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1781999999999997e-05 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.052103+0000", 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9087999999999999e-05 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.052758+0000", 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065510999999999998 2024-04-23T23:49:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.053172+0000", 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00041459500000000002 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.053202+0000", 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.053227+0000", 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4247999999999999e-05 2024-04-23T23:49:58.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1257 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.085337+0000", 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.259752712, 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025684800000000002, 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1257 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085337+0000", 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085337+0000", 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085338+0000", 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1939999999999999e-06 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085343+0000", 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.865e-06 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085354+0000", 2024-04-23T23:49:58.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085361+0000", 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085420+0000", 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9311000000000002e-05 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085496+0000", 2024-04-23T23:49:58.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5903999999999997e-05 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.085648+0000", 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015228900000000001 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.086079+0000", 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000431099 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.087807+0000", 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001727054 2024-04-23T23:49:58.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.087863+0000", 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.087905+0000", 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1687999999999998e-05 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1267 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.100066+0000", 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.24502363299999999, 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774890000000002, 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1267 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100066+0000", 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100066+0000", 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100067+0000", 2024-04-23T23:49:58.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3799999999999996e-07 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100072+0000", 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8279999999999999e-06 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100083+0000", 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100088+0000", 2024-04-23T23:49:58.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.101e-06 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100138+0000", 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9827999999999998e-05 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100219+0000", 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1324999999999998e-05 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100405+0000", 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018617899999999999 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.100875+0000", 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047007499999999997 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.102748+0000", 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001872498 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.102797+0000", 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9351000000000002e-05 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.102843+0000", 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6023000000000003e-05 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1277 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.110339+0000", 2024-04-23T23:49:58.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.23475016600000001, 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026947220000000001, 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1277 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110339+0000", 2024-04-23T23:49:58.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110339+0000", 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110340+0000", 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1299999999999999e-07 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110344+0000", 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7629999999999998e-06 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110354+0000", 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9639999999999998e-06 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110361+0000", 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0589999999999997e-06 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110418+0000", 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6773e-05 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110466+0000", 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8930000000000001e-05 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.110575+0000", 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010887600000000001 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.111020+0000", 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044466699999999999 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.112941+0000", 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001920974 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.112994+0000", 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3384000000000003e-05 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.113034+0000", 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9619000000000001e-05 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1313 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [stat,truncate 7077888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.151370+0000", 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.19371967700000001, 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013316109999999999, 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1313 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151370+0000", 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151370+0000", 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151371+0000", 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1999999999999998e-07 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151375+0000", 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.284e-06 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151385+0000", 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8090000000000004e-06 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151390+0000", 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151441+0000", 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1162999999999998e-05 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151498+0000", 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7111999999999998e-05 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.151581+0000", 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2898000000000005e-05 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.152228+0000", 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00064687700000000002 2024-04-23T23:49:58.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.152600+0000", 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037174099999999998 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.152653+0000", 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3987000000000003e-05 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.152701+0000", 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7849999999999998e-05 2024-04-23T23:49:58.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1382 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.221030+0000", 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.124059634, 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002676094, 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1382 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221030+0000", 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221030+0000", 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221031+0000", 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221035+0000", 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1540000000000004e-06 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221045+0000", 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0723999999999999e-05 2024-04-23T23:49:58.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221052+0000", 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221103+0000", 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0294000000000003e-05 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221152+0000", 2024-04-23T23:49:58.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9196999999999997e-05 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221267+0000", 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115401 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.221700+0000", 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00043272999999999999 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.223644+0000", 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001944016 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.223677+0000", 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2529e-05 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.223706+0000", 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9371000000000001e-05 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1441 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [truncate 6553600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.299720+0000", 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.045369077000000001, 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001390196, 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1441 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299720+0000", 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299720+0000", 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299722+0000", 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.39e-06 2024-04-23T23:49:58.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299728+0000", 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5830000000000003e-06 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299747+0000", 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8258999999999999e-05 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299756+0000", 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2690000000000005e-06 2024-04-23T23:49:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299820+0000", 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3956999999999998e-05 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299891+0000", 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1111000000000004e-05 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299976+0000", 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5082999999999998e-05 2024-04-23T23:49:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300993+0000", 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010165770000000001 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.301009+0000", 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6673000000000001e-05 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.301060+0000", 2024-04-23T23:49:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1258000000000003e-05 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.301111+0000", 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0036e-05 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1445 3.a e29/22 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head v 29'19, mlcod=29'17)", 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.300286+0000", 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.044803276000000003, 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031696720000000001, 2024-04-23T23:49:58.439 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300286+0000", 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300286+0000", 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300287+0000", 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9500000000000002e-07 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300292+0000", 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2249999999999999e-06 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300302+0000", 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8090000000000004e-06 2024-04-23T23:49:58.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300308+0000", 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8799999999999996e-06 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300364+0000", 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6755000000000001e-05 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:49:58.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300385+0000", 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0712000000000001e-05 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303422+0000", 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030370789999999998 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303456+0000", 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3617e-05 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.444 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:49:58.444 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:49:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:58.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:49:58.446 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:49:58.447 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:58.459+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:235 trunc from 0xe5fcbf9 to 0xb0e9752 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:236 read 0xa70de83 thru 0xa71c60d (0xe78b bytes) 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:239 write 0xbed1595 thru 0xbee0df6 (0xf862 bytes) 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:241 punch from 0x3e507ed to 0x3e5b0cf, (0xa8e2 bytes) 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:242 read 0x6535f55 thru 0x65419d3 (0xba7f bytes) 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:243 read 0x42a4bd thru 0x42ff6a (0x5aae bytes) 2024-04-23T23:49:58.447 INFO:teuthology.orchestra.run.smithi092.stdout:244 write 0xa3d0526 thru 0xa3d3e90 (0x396b bytes) 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:58.449 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:58.449 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:58.449 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:58.449 INFO:teuthology.orchestra.run.smithi092.stdout:245 trunc from 0xbee0df7 to 0x4133267 2024-04-23T23:49:58.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:58.450 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:58.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T23:49:58.451 INFO:teuthology.orchestra.run.smithi092.stdout:246 trunc from 0x4133267 to 0xd35d717 2024-04-23T23:49:58.451 INFO:teuthology.orchestra.run.smithi092.stdout:247 trunc from 0xd35d717 to 0xd888a79 2024-04-23T23:49:58.451 INFO:teuthology.orchestra.run.smithi092.stdout:248 write 0xe45c124 thru 0xe469378 (0xd255 bytes) 2024-04-23T23:49:58.451 INFO:teuthology.orchestra.run.smithi092.stdout:249 trunc from 0xe469379 to 0x546c60e 2024-04-23T23:49:58.452 INFO:teuthology.orchestra.run.smithi092.stdout:250 write 0xa94bd38 thru 0xa95217f (0x6448 bytes) 2024-04-23T23:49:58.454 INFO:teuthology.orchestra.run.smithi092.stdout:252 punch from 0x496740 to 0x49e743, (0x8003 bytes) 2024-04-23T23:49:58.454 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.454 INFO:teuthology.orchestra.run.smithi092.stdout:254 read 0x4daf959 thru 0x4db2adb (0x3183 bytes) 2024-04-23T23:49:58.454 INFO:teuthology.orchestra.run.smithi092.stdout:255 read 0x28374d1 thru 0x28422a8 (0xadd8 bytes) 2024-04-23T23:49:58.454 INFO:teuthology.orchestra.run.smithi092.stdout:256 read 0x64c6573 thru 0x64d64c3 (0xff51 bytes) 2024-04-23T23:49:58.455 INFO:teuthology.orchestra.run.smithi092.stdout:261 read 0x370624c thru 0x3709731 (0x34e6 bytes) 2024-04-23T23:49:58.455 INFO:teuthology.orchestra.run.smithi092.stdout:263 write 0x41be419 thru 0x41c80de (0x9cc6 bytes) 2024-04-23T23:49:58.459 INFO:teuthology.orchestra.run.smithi092.stdout:264 write 0x74870b2 thru 0x7491862 (0xa7b1 bytes) 2024-04-23T23:49:58.461 INFO:teuthology.orchestra.run.smithi092.stdout:266 punch from 0x9f67df2 to 0x9f6d0e0, (0x52ee bytes) 2024-04-23T23:49:58.461 INFO:teuthology.orchestra.run.smithi092.stdout:267 read 0x1cc6a26 thru 0x1ccdcd5 (0x72b0 bytes) 2024-04-23T23:49:58.467 INFO:teuthology.orchestra.run.smithi092.stdout:269 write 0xb27e079 thru 0xb27e8c3 (0x84b bytes) 2024-04-23T23:49:58.468 INFO:teuthology.orchestra.run.smithi092.stdout:272 read 0xddeb30 thru 0xdee3a5 (0xf876 bytes) 2024-04-23T23:49:58.469 INFO:teuthology.orchestra.run.smithi092.stdout:273 write 0x2c095ab thru 0x2c12c85 (0x96db bytes) 2024-04-23T23:49:58.471 INFO:teuthology.orchestra.run.smithi092.stdout:274 punch from 0x82b1b9a to 0x82c18fa, (0xfd60 bytes) 2024-04-23T23:49:58.479 INFO:teuthology.orchestra.run.smithi092.stdout:276 writesame 0xce7663f thru 0xce8338d data_size 0xd(0xcd4f bytes) 2024-04-23T23:49:58.479 INFO:teuthology.orchestra.run.smithi092.stdout:277 punch from 0x233ca9a to 0x2345c51, (0x91b7 bytes) 2024-04-23T23:49:58.479 INFO:teuthology.orchestra.run.smithi092.stdout:278 read 0x1c906af thru 0x1c94436 (0x3d88 bytes) 2024-04-23T23:49:58.479 INFO:teuthology.orchestra.run.smithi092.stdout:279 write 0xade1998 thru 0xade2309 (0x972 bytes) 2024-04-23T23:49:58.480 INFO:teuthology.orchestra.run.smithi092.stdout:280 punch from 0x8850c93 to 0x88580ba, (0x7427 bytes) 2024-04-23T23:49:58.481 INFO:teuthology.orchestra.run.smithi092.stdout:283 read 0x89304dd thru 0x893a2ac (0x9dd0 bytes) 2024-04-23T23:49:58.481 INFO:teuthology.orchestra.run.smithi092.stdout:284 write 0x4805793 thru 0x4806f32 (0x17a0 bytes) 2024-04-23T23:49:58.483 INFO:teuthology.orchestra.run.smithi092.stdout:286 trunc from 0xce8338e to 0xd748483 2024-04-23T23:49:58.488 INFO:teuthology.orchestra.run.smithi092.stdout:287 punch from 0xd4872b2 to 0xd48ddf6, (0x6b44 bytes) 2024-04-23T23:49:58.488 INFO:teuthology.orchestra.run.smithi092.stdout:288 write 0xd53cde thru 0xd5e5dc (0xa8ff bytes) 2024-04-23T23:49:58.490 INFO:teuthology.orchestra.run.smithi092.stdout:290 read 0xac1ac36 thru 0xac203a0 (0x576b bytes) 2024-04-23T23:49:58.490 INFO:teuthology.orchestra.run.smithi092.stdout:291 punch from 0x648f7b4 to 0x6491a48, (0x2294 bytes) 2024-04-23T23:49:58.490 INFO:teuthology.orchestra.run.smithi092.stdout:292 read 0xa06a04a thru 0xa075c8d (0xbc44 bytes) 2024-04-23T23:49:58.491 INFO:teuthology.orchestra.run.smithi092.stdout:296 write 0xcb65a76 thru 0xcb6fd96 (0xa321 bytes) 2024-04-23T23:49:58.495 INFO:teuthology.orchestra.run.smithi092.stdout:298 write 0x7e56ffb thru 0x7e589b8 (0x19be bytes) 2024-04-23T23:49:58.496 INFO:teuthology.orchestra.run.smithi092.stdout:300 read 0x666c666 thru 0x6675ea0 (0x983b bytes) 2024-04-23T23:49:58.496 INFO:teuthology.orchestra.run.smithi092.stdout:301 write 0xcc0baa9 thru 0xcc1b75f (0xfcb7 bytes) 2024-04-23T23:49:58.500 INFO:teuthology.orchestra.run.smithi092.stdout:302 write 0x75fdb3a thru 0x7608ea8 (0xb36f bytes) 2024-04-23T23:49:58.501 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.501 INFO:teuthology.orchestra.run.smithi092.stdout:304 punch from 0x865c71e to 0x8662400, (0x5ce2 bytes) 2024-04-23T23:49:58.501 INFO:teuthology.orchestra.run.smithi092.stdout:305 write 0x9a678d0 thru 0x9a680cb (0x7fc bytes) 2024-04-23T23:49:58.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:49:58.511 INFO:teuthology.orchestra.run.smithi092.stdout:307 trunc from 0xd748483 to 0x6d0ee9c 2024-04-23T23:49:58.542 INFO:teuthology.orchestra.run.smithi092.stdout:308 write 0xadb7abe thru 0xadbc006 (0x4549 bytes) 2024-04-23T23:49:58.547 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:58.560+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:58.547 INFO:teuthology.orchestra.run.smithi092.stdout:309 read 0x66d8d0 thru 0x67c364 (0xea95 bytes) 2024-04-23T23:49:58.547 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.547 INFO:teuthology.orchestra.run.smithi092.stdout:313 read 0x1b4ce06 thru 0x1b51614 (0x480f bytes) 2024-04-23T23:49:58.548 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.557 INFO:teuthology.orchestra.run.smithi092.stdout:315 writesame 0xc86fd52 thru 0xc87f349 data_size 0x44(0xf5f8 bytes) 2024-04-23T23:49:58.557 INFO:teuthology.orchestra.run.smithi092.stdout:316 write 0x3e4ac53 thru 0x3e52435 (0x77e3 bytes) 2024-04-23T23:49:58.569 INFO:teuthology.orchestra.run.smithi092.stdout:317 write 0xe6d9b9f thru 0xe6e5411 (0xb873 bytes) 2024-04-23T23:49:58.573 INFO:teuthology.orchestra.run.smithi092.stdout:318 read 0x7058ae5 thru 0x70611a0 (0x86bc bytes) 2024-04-23T23:49:58.574 INFO:teuthology.orchestra.run.smithi092.stdout:319 punch from 0x431b101 to 0x43204cb, (0x53ca bytes) 2024-04-23T23:49:58.574 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.574 INFO:teuthology.orchestra.run.smithi092.stdout:321 write 0x283f1a0 thru 0x283f660 (0x4c1 bytes) 2024-04-23T23:49:58.576 INFO:teuthology.orchestra.run.smithi092.stdout:322 write 0x3a92680 thru 0x3a9ee0d (0xc78e bytes) 2024-04-23T23:49:58.578 INFO:teuthology.orchestra.run.smithi092.stdout:325 punch from 0x7c93300 to 0x7c958c7, (0x25c7 bytes) 2024-04-23T23:49:58.578 INFO:teuthology.orchestra.run.smithi092.stdout:326 punch from 0x3d65569 to 0x3d6cb80, (0x7617 bytes) 2024-04-23T23:49:58.578 INFO:teuthology.orchestra.run.smithi092.stdout:328 write 0x86fe9a1 thru 0x8709466 (0xaac6 bytes) 2024-04-23T23:49:58.579 INFO:teuthology.orchestra.run.smithi092.stdout:329 read 0xadf015d thru 0xadf3fcb (0x3e6f bytes) 2024-04-23T23:49:58.579 INFO:teuthology.orchestra.run.smithi092.stdout:330 read 0x9efa990 thru 0x9f0807e (0xd6ef bytes) 2024-04-23T23:49:58.595 INFO:teuthology.orchestra.run.smithi092.stdout:331 trunc from 0xe6e5412 to 0x1f3365e 2024-04-23T23:49:58.595 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:49:58.595 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:58.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:58.610 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:58.610 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:58.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T23:49:58.615 INFO:teuthology.orchestra.run.smithi092.stdout:332 read 0x62c3d7 thru 0x63916a (0xcd94 bytes) 2024-04-23T23:49:58.631 INFO:teuthology.orchestra.run.smithi092.stdout:333 writesame 0x7f6f75d thru 0x7f7d323 data_size 0x1(0xdbc7 bytes) 2024-04-23T23:49:58.632 INFO:teuthology.orchestra.run.smithi092.stdout:334 read 0x2069fe8 thru 0x207813b (0xe154 bytes) 2024-04-23T23:49:58.633 INFO:teuthology.orchestra.run.smithi092.stdout:336 writesame 0x737994 thru 0x7396b1 data_size 0x1(0x1d1e bytes) 2024-04-23T23:49:58.640 INFO:teuthology.orchestra.run.smithi092.stdout:338 writesame 0x427ba26 thru 0x428887e data_size 0x1(0xce59 bytes) 2024-04-23T23:49:58.648 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:58.661+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:58.658 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:49:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:49:58.678 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:49:58.678 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:49:58.682 INFO:teuthology.orchestra.run.smithi092.stdout:341 write 0x70160c1 thru 0x701a680 (0x45c0 bytes) 2024-04-23T23:49:58.684 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.684 INFO:teuthology.orchestra.run.smithi092.stdout:344 read 0x351c245 thru 0x3522a3b (0x67f7 bytes) 2024-04-23T23:49:58.685 INFO:teuthology.orchestra.run.smithi092.stdout:345 read 0x5ab0833 thru 0x5abfa85 (0xf253 bytes) 2024-04-23T23:49:58.686 INFO:teuthology.orchestra.run.smithi092.stdout:346 read 0xa78306 thru 0xa7b2bd (0x2fb8 bytes) 2024-04-23T23:49:58.686 INFO:teuthology.orchestra.run.smithi092.stdout:347 write 0x547311a thru 0x547459e (0x1485 bytes) 2024-04-23T23:49:58.688 INFO:teuthology.orchestra.run.smithi092.stdout:348 write 0x675ddcc thru 0x67682d2 (0xa507 bytes) 2024-04-23T23:49:58.689 INFO:teuthology.orchestra.run.smithi092.stdout:350 punch from 0x4f1dbdf to 0x4f2b780, (0xdba1 bytes) 2024-04-23T23:49:58.697 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:49:58.697 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:49:58.699 INFO:teuthology.orchestra.run.smithi092.stdout:351 trunc from 0x7f7d324 to 0x4be0c7 2024-04-23T23:49:58.718 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:49:58.718 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:49:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:353 writesame 0xdbbb965 thru 0xdbc485c data_size 0x78(0x8ef8 bytes) 2024-04-23T23:49:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:354 read 0x4bfa9d0 thru 0x4bfb0bb (0x6ec bytes) 2024-04-23T23:49:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:356 punch from 0x85b19e4 to 0x85b64ed, (0x4b09 bytes) 2024-04-23T23:49:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe70b449 2024-04-23T23:49:58.736 INFO:teuthology.orchestra.run.smithi092.stdout:359 trunc from 0xdbc485d to 0xe70b449 2024-04-23T23:49:58.736 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:49:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:49:58.742 INFO:teuthology.orchestra.run.smithi092.stdout:360 writesame 0x37d3a6f thru 0x37e1d4b data_size 0x51(0xe2dd bytes) 2024-04-23T23:49:58.744 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.744 INFO:teuthology.orchestra.run.smithi092.stdout:362 read 0xc506137 thru 0xc50a494 (0x435e bytes) 2024-04-23T23:49:58.744 INFO:teuthology.orchestra.run.smithi092.stdout:363 writesame 0xccd8b3e thru 0xccdb3f1 data_size 0xa(0x28b4 bytes) 2024-04-23T23:49:58.748 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:58.762+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:58.748 INFO:teuthology.orchestra.run.smithi092.stdout:364 writesame 0x857b538 thru 0x8580dc1 data_size 0x7(0x588a bytes) 2024-04-23T23:49:58.751 INFO:teuthology.orchestra.run.smithi092.stdout:365 write 0xd41f767 thru 0xd427396 (0x7c30 bytes) 2024-04-23T23:49:58.752 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:49:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:49:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:49:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:49:58.752 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:49:58.756 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:49:58.756 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:49:58.758 INFO:teuthology.orchestra.run.smithi092.stdout:367 write 0x59ca45 thru 0x59d72e (0xcea bytes) 2024-04-23T23:49:58.759 INFO:teuthology.orchestra.run.smithi092.stdout:375 write 0xaa95e41 thru 0xaaa143d (0xb5fd bytes) 2024-04-23T23:49:58.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:49:58.763 INFO:teuthology.orchestra.run.smithi092.stdout:376 read 0x7022d4d thru 0x70267b4 (0x3a68 bytes) 2024-04-23T23:49:58.763 INFO:teuthology.orchestra.run.smithi092.stdout:377 writesame 0x74b956 thru 0x754e80 data_size 0x3(0x952b bytes) 2024-04-23T23:49:58.774 INFO:teuthology.orchestra.run.smithi092.stdout:378 read 0xa2ccf54 thru 0xa2d3765 (0x6812 bytes) 2024-04-23T23:49:58.774 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:49:58.775 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:49:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:380 write 0x1478521 thru 0x1483467 (0xaf47 bytes) 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:58.781 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:58.782 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:58.782 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:58.782 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:58.786 INFO:teuthology.orchestra.run.smithi092.stdout:381 trunc from 0xe70b449 to 0x6b98d95 2024-04-23T23:49:58.786 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:58.788 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:58.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-04-23T23:49:58.794 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:49:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:49:58.805 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.805 INFO:teuthology.orchestra.run.smithi092.stdout:383 read 0x8963a9 thru 0x89ffa8 (0x9c00 bytes) 2024-04-23T23:49:58.806 INFO:teuthology.orchestra.run.smithi092.stdout:384 read 0x33f5d62 thru 0x33fbc5d (0x5efc bytes) 2024-04-23T23:49:58.819 INFO:teuthology.orchestra.run.smithi092.stdout:385 writesame 0xbdf95bb thru 0xbe00f62 data_size 0x44(0x79a8 bytes) 2024-04-23T23:49:58.825 INFO:teuthology.orchestra.run.smithi092.stdout:386 writesame 0xc59078a thru 0xc5958ff data_size 0x1(0x5176 bytes) 2024-04-23T23:49:58.825 INFO:teuthology.orchestra.run.smithi092.stdout:388 write 0x86f64c0 thru 0x87046d1 (0xe212 bytes) 2024-04-23T23:49:58.833 INFO:teuthology.orchestra.run.smithi092.stdout:389 trunc from 0xc595900 to 0x7309948 2024-04-23T23:49:58.842 INFO:teuthology.orchestra.run.smithi092.stdout:390 write 0x2698122 thru 0x269a386 (0x2265 bytes) 2024-04-23T23:49:58.843 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.843 INFO:teuthology.orchestra.run.smithi092.stdout:392 read 0x4fdb95 thru 0x50447e (0x68ea bytes) 2024-04-23T23:49:58.849 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:49:58.863+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:49:58.851 INFO:teuthology.orchestra.run.smithi092.stdout:393 write 0x9687842 thru 0x969040a (0x8bc9 bytes) 2024-04-23T23:49:58.855 INFO:teuthology.orchestra.run.smithi092.stdout:395 write 0xad2388 thru 0xae0fdc (0xec55 bytes) 2024-04-23T23:49:58.856 INFO:teuthology.orchestra.run.smithi092.stdout:396 writesame 0x62ccf09 thru 0x62d4d3f data_size 0x1(0x7e37 bytes) 2024-04-23T23:49:58.893 INFO:teuthology.orchestra.run.smithi092.stdout:397 write 0xeca65a8 thru 0xecacd0f (0x6768 bytes) 2024-04-23T23:49:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:401 punch from 0xbf2414 to 0xc00caf, (0xe89b bytes) 2024-04-23T23:49:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:402 read 0xe94a19b thru 0xe958473 (0xe2d9 bytes) 2024-04-23T23:49:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:403 read 0x91f08e4 thru 0x91fc4ef (0xbc0c bytes) 2024-04-23T23:49:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:404 writesame 0x5356896 thru 0x53647e4 data_size 0x1(0xdf4f bytes) 2024-04-23T23:49:58.907 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:58.907 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:58.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:49:58.933 INFO:teuthology.orchestra.run.smithi092.stdout:405 write 0x74c810 thru 0x75214f (0x5940 bytes) 2024-04-23T23:49:58.936 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:58.936 INFO:teuthology.orchestra.run.smithi092.stdout:407 writesame 0x5d25a06 thru 0x5d2bcbf data_size 0x1(0x62ba bytes) 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:58.947 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:58.948 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:58.948 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:58.948 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:58.949 INFO:teuthology.orchestra.run.smithi092.stdout:408 punch from 0xce4f7a2 to 0xce5c5c2, (0xce20 bytes) 2024-04-23T23:49:58.949 INFO:teuthology.orchestra.run.smithi092.stdout:409 read 0x9b1bb10 thru 0x9b25d74 (0xa265 bytes) 2024-04-23T23:49:58.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:58.951 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:58.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-04-23T23:49:58.960 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:58.974+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:58.963 INFO:teuthology.orchestra.run.smithi092.stdout:410 trunc from 0xecacd10 to 0x4183b76 2024-04-23T23:49:58.985 INFO:teuthology.orchestra.run.smithi092.stdout:411 trunc from 0x4183b76 to 0x795e5a5 2024-04-23T23:49:58.991 INFO:teuthology.orchestra.run.smithi092.stdout:412 clone 4 order 23 su 262144 sc 3 2024-04-23T23:49:59.020 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.021 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:49:59.050 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:59.064+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:59.095 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:59.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:59.095 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:59.095 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:59.095 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:59.098 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:59.098 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:59.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-04-23T23:49:59.172 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.172 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:49:59.184 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:59.198+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:59.251 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:49:59.265+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:49:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:59.273 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:59.274 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:59.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-04-23T23:49:59.285 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.285 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:49:59.337 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:49:59.378 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:59.392+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:59.441 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.441 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:49:59.453 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:49:59.466+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:49:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:49:59.456 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:49:59.458 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:49:59.459 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:49:59.487 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-04-23T23:49:59.554 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:59.567+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:59.594 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.594 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:49:59.655 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:49:59.668+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:49:59.704 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.705 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:49:59.758 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:59.772+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:59.775 INFO:teuthology.orchestra.run.smithi092.stdout:413 write 0x37db579 thru 0x37ea5ca (0xf052 bytes) 2024-04-23T23:49:59.793 INFO:teuthology.orchestra.run.smithi092.stdout:414 writesame 0xb6501bf thru 0xb653ef4 data_size 0x5(0x3d36 bytes) 2024-04-23T23:49:59.794 INFO:teuthology.orchestra.run.smithi092.stdout:415 writesame 0x21f1b1a thru 0x21fa086 data_size 0x1(0x856d bytes) 2024-04-23T23:49:59.812 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.813 INFO:teuthology.orchestra.run.smithi092.stdout:417 punch from 0xa06b285 to 0xa0778ec, (0xc667 bytes) 2024-04-23T23:49:59.813 INFO:teuthology.orchestra.run.smithi092.stdout:418 read 0x21e79a7 thru 0x21ebeed (0x4547 bytes) 2024-04-23T23:49:59.822 INFO:teuthology.orchestra.run.smithi092.stdout:420 writesame 0xd9f0ddd thru 0xd9fedb6 data_size 0x3(0xdfda bytes) 2024-04-23T23:49:59.822 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.822 INFO:teuthology.orchestra.run.smithi092.stdout:425 write 0xa0bf28f thru 0xa0cb29d (0xc00f bytes) 2024-04-23T23:49:59.825 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.825 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.826 INFO:teuthology.orchestra.run.smithi092.stdout:426 write 0x865da15 thru 0x866bc80 (0xe26c bytes) 2024-04-23T23:49:59.830 INFO:teuthology.orchestra.run.smithi092.stdout:427 read 0x1793c43 thru 0x17a2e42 (0xf200 bytes) 2024-04-23T23:49:59.830 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.830 INFO:teuthology.orchestra.run.smithi092.stdout:430 writesame 0x4b01e10 thru 0x4b08ca8 data_size 0x1(0x6e99 bytes) 2024-04-23T23:49:59.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:49:59.846 INFO:teuthology.orchestra.run.smithi092.stdout:431 writesame 0x36f4dfa thru 0x36f69e0 data_size 0x1(0x1be7 bytes) 2024-04-23T23:49:59.857 INFO:teuthology.orchestra.run.smithi092.stdout:432 read 0x89bc1c5 thru 0x89c6830 (0xa66c bytes) 2024-04-23T23:49:59.857 INFO:teuthology.orchestra.run.smithi092.stdout:433 read 0xacce0e0 thru 0xacd3444 (0x5365 bytes) 2024-04-23T23:49:59.857 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:49:59.871+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:49:59.863 INFO:teuthology.orchestra.run.smithi092.stdout:434 trunc from 0xd9fedb7 to 0x8289b18 2024-04-23T23:49:59.881 INFO:teuthology.orchestra.run.smithi092.stdout:435 read 0x5d2fee4 thru 0x5d37888 (0x79a5 bytes) 2024-04-23T23:49:59.887 INFO:teuthology.orchestra.run.smithi092.stdout:439 write 0x8705024 thru 0x870a9db (0x59b8 bytes) 2024-04-23T23:49:59.890 INFO:teuthology.orchestra.run.smithi092.stdout:442 trunc from 0x870a9dc to 0xa543c62 2024-04-23T23:49:59.907 INFO:teuthology.orchestra.run.smithi092.stdout:443 write 0xe8dd658 thru 0xe8e5e01 (0x87aa bytes) 2024-04-23T23:49:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:445 read 0x71e0249 thru 0x71ecab3 (0xc86b bytes) 2024-04-23T23:49:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:446 write 0x63a807b thru 0x63ac4e4 (0x446a bytes) 2024-04-23T23:49:59.914 INFO:teuthology.orchestra.run.smithi092.stdout:447 read 0x465e8ad thru 0x466dd1b (0xf46f bytes) 2024-04-23T23:49:59.914 INFO:teuthology.orchestra.run.smithi092.stdout:448 writesame 0xe42c742 thru 0xe434926 data_size 0x1(0x81e5 bytes) 2024-04-23T23:49:59.936 INFO:teuthology.orchestra.run.smithi092.stdout:449 punch from 0xa59240c to 0xa5a17e0, (0xf3d4 bytes) 2024-04-23T23:49:59.936 INFO:teuthology.orchestra.run.smithi092.stdout:451 read 0x3335ec4 thru 0x3337aca (0x1c07 bytes) 2024-04-23T23:49:59.937 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.937 INFO:teuthology.orchestra.run.smithi092.stdout:453 write 0x8c28177 thru 0x8c34753 (0xc5dd bytes) 2024-04-23T23:49:59.948 INFO:teuthology.orchestra.run.smithi092.stdout:454 trunc from 0xe8e5e02 to 0x65a7833 2024-04-23T23:49:59.959 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:49:59.971+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:49:59.968 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:49:59.968 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:49:59.969 INFO:teuthology.orchestra.run.smithi092.stdout:455 read 0x4713b6 thru 0x476635 (0x5280 bytes) 2024-04-23T23:49:59.970 INFO:teuthology.orchestra.run.smithi092.stdout:457 write 0xc5ef6e thru 0xc6bc7b (0xcd0e bytes) 2024-04-23T23:49:59.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:49:59.979 INFO:teuthology.orchestra.run.smithi092.stdout:458 punch from 0x1d87607 to 0x1d8ec53, (0x764c bytes) 2024-04-23T23:49:59.979 INFO:teuthology.orchestra.run.smithi092.stdout:459 read 0xa7b291 thru 0xa86960 (0xb6d0 bytes) 2024-04-23T23:49:59.979 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.984 INFO:teuthology.orchestra.run.smithi092.stdout:461 trunc from 0x65a7833 to 0xa6df3a9 2024-04-23T23:49:59.990 INFO:teuthology.orchestra.run.smithi092.stdout:462 punch from 0x14d4c4d to 0x14d4e4b, (0x1fe bytes) 2024-04-23T23:49:59.990 INFO:teuthology.orchestra.run.smithi092.stdout:465 write 0x523ce38 thru 0x5247ac7 (0xac90 bytes) 2024-04-23T23:49:59.997 INFO:teuthology.orchestra.run.smithi092.stdout:466 read 0x584b29b thru 0x5851144 (0x5eaa bytes) 2024-04-23T23:49:59.997 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:49:59.997 INFO:teuthology.orchestra.run.smithi092.stdout:468 read 0x3a1af6 thru 0x3a8e4d (0x7358 bytes) 2024-04-23T23:49:59.997 INFO:teuthology.orchestra.run.smithi092.stdout:469 punch from 0x609a3c2 to 0x609e6e4, (0x4322 bytes) 2024-04-23T23:50:00.007 INFO:teuthology.orchestra.run.smithi092.stdout:471 writesame 0xacbc66d thru 0xacc9585 data_size 0x1(0xcf19 bytes) 2024-04-23T23:50:00.007 INFO:teuthology.orchestra.run.smithi092.stdout:472 punch from 0x42dda1e to 0x42e8ab1, (0xb093 bytes) 2024-04-23T23:50:00.007 INFO:teuthology.orchestra.run.smithi092.stdout:473 punch from 0x549983e to 0x549f2da, (0x5a9c bytes) 2024-04-23T23:50:00.007 INFO:teuthology.orchestra.run.smithi092.stdout:474 punch from 0x8147cda to 0x81552b8, (0xd5de bytes) 2024-04-23T23:50:00.013 INFO:teuthology.orchestra.run.smithi092.stdout:476 trunc from 0xacc9586 to 0x53ff76f 2024-04-23T23:50:00.025 INFO:teuthology.orchestra.run.smithi092.stdout:477 read 0x411753a thru 0x412652f (0xeff6 bytes) 2024-04-23T23:50:00.025 INFO:teuthology.orchestra.run.smithi092.stdout:478 punch from 0x2ccf802 to 0x2cd5255, (0x5a53 bytes) 2024-04-23T23:50:00.025 INFO:teuthology.orchestra.run.smithi092.stdout:479 write 0x112d36c thru 0x11342e0 (0x6f75 bytes) 2024-04-23T23:50:00.033 INFO:teuthology.orchestra.run.smithi092.stdout:480 read 0x4b10d90 thru 0x4b15612 (0x4883 bytes) 2024-04-23T23:50:00.034 INFO:teuthology.orchestra.run.smithi092.stdout:481 punch from 0x11be145 to 0x11c521f, (0x70da bytes) 2024-04-23T23:50:00.034 INFO:teuthology.orchestra.run.smithi092.stdout:482 write 0x450a834 thru 0x45168b7 (0xc084 bytes) 2024-04-23T23:50:00.048 INFO:teuthology.orchestra.run.smithi092.stdout:483 writesame 0xb987bd7 thru 0xb990b11 data_size 0x1(0x8f3b bytes) 2024-04-23T23:50:00.048 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:00.048 INFO:teuthology.orchestra.run.smithi092.stdout:485 write 0x446d846 thru 0x447c10b (0xe8c6 bytes) 2024-04-23T23:50:00.049 INFO:teuthology.orchestra.run.smithi092.stdout:486 writesame 0xb28e896 thru 0xb2947ad data_size 0x22(0x5f18 bytes) 2024-04-23T23:50:00.053 INFO:teuthology.orchestra.run.smithi092.stdout:487 write 0xfcc51 thru 0x102e57 (0x6207 bytes) 2024-04-23T23:50:00.058 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:00.072+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:00.061 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:00.061 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:00.062 INFO:teuthology.orchestra.run.smithi092.stdout:491 write 0xb06e112 thru 0xb074b37 (0x6a26 bytes) 2024-04-23T23:50:00.063 INFO:teuthology.orchestra.run.smithi092.stdout:492 read 0x20b15a5 thru 0x20bf76c (0xe1c8 bytes) 2024-04-23T23:50:00.063 INFO:teuthology.orchestra.run.smithi092.stdout:493 punch from 0x174cd67 to 0x1751ec0, (0x5159 bytes) 2024-04-23T23:50:00.064 INFO:teuthology.orchestra.run.smithi092.stdout:494 punch from 0x754c0fd to 0x755a32a, (0xe22d bytes) 2024-04-23T23:50:00.064 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe9a39d1 2024-04-23T23:50:00.067 INFO:teuthology.orchestra.run.smithi092.stdout:495 trunc from 0xb990b12 to 0xe9a39d1 2024-04-23T23:50:00.072 INFO:teuthology.orchestra.run.smithi092.stdout:496 write 0xc1bcb95 thru 0xc1cb255 (0xe6c1 bytes) 2024-04-23T23:50:00.076 INFO:teuthology.orchestra.run.smithi092.stdout:497 write 0x12fd499 thru 0x1309d98 (0xc900 bytes) 2024-04-23T23:50:00.078 INFO:teuthology.orchestra.run.smithi092.stdout:498 write 0x7dea498 thru 0x7df2fb3 (0x8b1c bytes) 2024-04-23T23:50:00.080 INFO:teuthology.orchestra.run.smithi092.stdout:499 write 0x864245b thru 0x864b58a (0x9130 bytes) 2024-04-23T23:50:00.083 INFO:teuthology.orchestra.run.smithi092.stdout:500 punch from 0x9cacaa4 to 0x9cb797f, (0xaedb bytes) 2024-04-23T23:50:00.083 INFO:teuthology.orchestra.run.smithi092.stdout:502 write 0x4d79537 thru 0x4d861c8 (0xcc92 bytes) 2024-04-23T23:50:00.085 INFO:teuthology.orchestra.run.smithi092.stdout:503 clone 5 order 25 su 1048576 sc 5 2024-04-23T23:50:00.097 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:00.097 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:00.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:00.190 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:00.204+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:00.250 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:00.250 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:00.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:00.299 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:00.313+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:00.360 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:00.374+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:00.405 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone4 2024-04-23T23:50:00.408 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:00.408 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:00.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:00.461 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:00.475+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:00.562 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:00.576+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:00.569 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:00.569 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:00.569 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-04-23T23:50:00.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:00.664 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:00.677+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:00.733 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:00.733 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:00.737 INFO:teuthology.orchestra.run.smithi092.stdout:505 write 0xedaa928 thru 0xedb95ad (0xec86 bytes) 2024-04-23T23:50:00.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:00.743 INFO:teuthology.orchestra.run.smithi092.stdout:506 read 0x1ad1625 thru 0x1ad1867 (0x243 bytes) 2024-04-23T23:50:00.743 INFO:teuthology.orchestra.run.smithi092.stdout:507 write 0x759c892 thru 0x75a329a (0x6a09 bytes) 2024-04-23T23:50:00.750 INFO:teuthology.orchestra.run.smithi092.stdout:509 punch from 0xaa474c9 to 0xaa554f0, (0xe027 bytes) 2024-04-23T23:50:00.750 INFO:teuthology.orchestra.run.smithi092.stdout:510 punch from 0x6d48137 to 0x6d4ffdb, (0x7ea4 bytes) 2024-04-23T23:50:00.750 INFO:teuthology.orchestra.run.smithi092.stdout:512 read 0xd4caaa0 thru 0xd4cf126 (0x4687 bytes) 2024-04-23T23:50:00.751 INFO:teuthology.orchestra.run.smithi092.stdout:513 write 0x5a67fb2 thru 0x5a6ca49 (0x4a98 bytes) 2024-04-23T23:50:00.762 INFO:teuthology.orchestra.run.smithi092.stdout:514 write 0xd1f7fc0 thru 0xd1fed7b (0x6dbc bytes) 2024-04-23T23:50:00.764 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:00.778+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:00.767 INFO:teuthology.orchestra.run.smithi092.stdout:515 read 0x78b1c8e thru 0x78c0282 (0xe5f5 bytes) 2024-04-23T23:50:00.768 INFO:teuthology.orchestra.run.smithi092.stdout:517 punch from 0x45f04e6 to 0x45f251e, (0x2038 bytes) 2024-04-23T23:50:00.768 INFO:teuthology.orchestra.run.smithi092.stdout:518 read 0x3d27559 thru 0x3d28c17 (0x16bf bytes) 2024-04-23T23:50:00.768 INFO:teuthology.orchestra.run.smithi092.stdout:519 read 0x4497c64 thru 0x449c380 (0x471d bytes) 2024-04-23T23:50:00.768 INFO:teuthology.orchestra.run.smithi092.stdout:520 clone 6 order 22 su 524288 sc 15 2024-04-23T23:50:00.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:00.849 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:00.850 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:00.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:00.865 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:00.879+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:00.930 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:00.931 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:00.948 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:00.948 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:00.965 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:00.965 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:00.966 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:00.979+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:00.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:00.982 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:00.999 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:00.999 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:313 3.0 e28/22 3:04479396:::rbd_id.image_client.0-clone2:head v 28'1, mlcod=0'0)", 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6410584850000003, 2024-04-23T23:50:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082763200000000002, 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:50:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378180+0000", 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.91e-07 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378186+0000", 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1020000000000002e-06 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378197+0000", 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.061e-05 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378206+0000", 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1400000000000006e-06 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378280+0000", 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3749000000000002e-05 2024-04-23T23:50:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378307+0000", 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7475999999999999e-05 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.378954+0000", 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064691200000000003 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.379008+0000", 2024-04-23T23:50:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3352e-05 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:536 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'1, mlcod=0'0)", 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.881317+0000", 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1379219029999996, 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00077659799999999998, 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881317+0000", 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881317+0000", 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881318+0000", 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.288e-06 2024-04-23T23:50:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881347+0000", 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8812000000000001e-05 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881364+0000", 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7028999999999999e-05 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881374+0000", 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7610000000000002e-06 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881432+0000", 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8906999999999997e-05 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.881453+0000", 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0421e-05 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.882062+0000", 2024-04-23T23:50:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060873000000000003 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.882093+0000", 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1649999999999997e-05 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:541 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'2, mlcod=0'0)", 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.894384+0000", 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1248548730000003, 2024-04-23T23:50:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071852400000000001, 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894384+0000", 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894384+0000", 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894385+0000", 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894392+0000", 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5900000000000002e-06 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894406+0000", 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4062e-05 2024-04-23T23:50:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894418+0000", 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1841000000000001e-05 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894488+0000", 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9948000000000002e-05 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.894508+0000", 2024-04-23T23:50:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9531999999999999e-05 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895055+0000", 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054707200000000001 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.895102+0000", 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7401999999999998e-05 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:566 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'3, mlcod=28'1)", 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1037116300000003, 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001175798, 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915528+0000", 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0580000000000001e-06 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915539+0000", 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638999999999999e-05 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915554+0000", 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5141e-05 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915565+0000", 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0789e-05 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915620+0000", 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5189999999999998e-05 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915642+0000", 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2262000000000001e-05 2024-04-23T23:50:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916672+0000", 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010296109999999999 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916703+0000", 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1108000000000001e-05 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1147 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'4, mlcod=28'2)", 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.0876274989999999, 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022957910000000002, 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931628+0000", 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6813999999999999e-05 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931636+0000", 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931644+0000", 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3359999999999997e-06 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931703+0000", 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9598000000000001e-05 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931724+0000", 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0735000000000001e-05 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933865+0000", 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002141165 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933907+0000", 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1768999999999999e-05 2024-04-23T23:50:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1174 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'5, mlcod=28'3)", 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.957518+0000", 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.0617206179999998, 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078874000000000001, 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957518+0000", 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957518+0000", 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957519+0000", 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0599999999999999e-07 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957544+0000", 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5191999999999999e-05 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957556+0000", 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1753000000000001e-05 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957564+0000", 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8029999999999997e-06 2024-04-23T23:50:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957625+0000", 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1552999999999998e-05 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.957648+0000", 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2813999999999999e-05 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.958258+0000", 2024-04-23T23:50:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060963800000000002 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.958307+0000", 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9081e-05 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1197 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'6, mlcod=29'4)", 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.017150+0000", 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.0020885609999999, 2024-04-23T23:50:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070550099999999998, 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017150+0000", 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017150+0000", 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017151+0000", 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.172e-06 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017162+0000", 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1156e-05 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017180+0000", 2024-04-23T23:50:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7958e-05 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017190+0000", 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0243e-05 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017258+0000", 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7958000000000002e-05 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017281+0000", 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2535999999999999e-05 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017819+0000", 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053817000000000003 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.017855+0000", 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6307999999999999e-05 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1212 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'7, mlcod=29'5)", 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.026896+0000", 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9923422940000002, 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066694300000000003, 2024-04-23T23:50:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026896+0000", 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026896+0000", 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026897+0000", 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8999999999999996e-07 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026910+0000", 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2884999999999999e-05 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026921+0000", 2024-04-23T23:50:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026927+0000", 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8710000000000002e-06 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026981+0000", 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3908e-05 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.026998+0000", 2024-04-23T23:50:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6469000000000001e-05 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.027527+0000", 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052894300000000003 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.027563+0000", 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6504000000000002e-05 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1237 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'8, mlcod=29'6)", 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.054237+0000", 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.965001886, 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000775883, 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054237+0000", 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054237+0000", 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054238+0000", 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054281+0000", 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3275999999999997e-05 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054298+0000", 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6759e-05 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054307+0000", 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6109999999999994e-06 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054359+0000", 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2930999999999999e-05 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054373+0000", 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3715e-05 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.054975+0000", 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060198900000000004 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.055013+0000", 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7388999999999997e-05 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1254 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'9, mlcod=29'7)", 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.080383+0000", 2024-04-23T23:50:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9388551400000003, 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071288799999999995, 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080383+0000", 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080383+0000", 2024-04-23T23:50:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080385+0000", 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1990000000000001e-06 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080394+0000", 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080413+0000", 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8853999999999999e-05 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080424+0000", 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0798000000000001e-05 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080492+0000", 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7960999999999995e-05 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080516+0000", 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4219999999999999e-05 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.081065+0000", 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054946499999999996 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.081096+0000", 2024-04-23T23:50:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0809999999999998e-05 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1275 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'10, mlcod=29'8)", 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.105488+0000", 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9137509210000001, 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000727179, 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105488+0000", 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105488+0000", 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105489+0000", 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6599999999999994e-07 2024-04-23T23:50:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105525+0000", 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6767000000000002e-05 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105539+0000", 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3701e-05 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105545+0000", 2024-04-23T23:50:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4439999999999996e-06 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105601+0000", 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5649000000000002e-05 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.105618+0000", 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6534000000000001e-05 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.106172+0000", 2024-04-23T23:50:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055470499999999996 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.106215+0000", 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2413000000000002e-05 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1287 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'11, mlcod=29'9)", 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.120327+0000", 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.8989119840000002, 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00079078000000000004, 2024-04-23T23:50:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120327+0000", 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120327+0000", 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120328+0000", 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120371+0000", 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3133000000000002e-05 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120385+0000", 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.378e-05 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120391+0000", 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120470+0000", 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9083000000000002e-05 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.120486+0000", 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5328999999999998e-05 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.121087+0000", 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060143000000000002 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.121117+0000", 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.012e-05 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1316 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'12, mlcod=29'10)", 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.152302+0000", 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.866936205, 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00079394899999999996, 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152302+0000", 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152302+0000", 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152303+0000", 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.158e-06 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152318+0000", 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4647000000000001e-05 2024-04-23T23:50:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152338+0000", 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9862000000000001e-05 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152348+0000", 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152411+0000", 2024-04-23T23:50:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3009000000000006e-05 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.152431+0000", 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0210000000000001e-05 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.153045+0000", 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000613734 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.153096+0000", 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1172999999999999e-05 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1444 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'14, mlcod=29'12)", 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.300209+0000", 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7190299160000002, 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074046099999999998, 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300209+0000", 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300209+0000", 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300210+0000", 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.141e-06 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300219+0000", 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2529999999999993e-06 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300237+0000", 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8304e-05 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300247+0000", 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9499999999999996e-06 2024-04-23T23:50:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300313+0000", 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5733000000000002e-05 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300335+0000", 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1579000000000001e-05 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300905+0000", 2024-04-23T23:50:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000570373 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.300949+0000", 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4128000000000002e-05 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1471 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'15, mlcod=29'13)", 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.325615+0000", 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6936233199999999, 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068208699999999995, 2024-04-23T23:50:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325615+0000", 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325615+0000", 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325616+0000", 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.95e-07 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325637+0000", 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325651+0000", 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4151000000000001e-05 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325657+0000", 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3980000000000004e-06 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325716+0000", 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9175000000000002e-05 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.325731+0000", 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4876e-05 2024-04-23T23:50:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.326267+0000", 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053552299999999999 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.326297+0000", 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0641e-05 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1526 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'16, mlcod=29'14)", 2024-04-23T23:50:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6335577020000001, 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021410930000000002, 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385682+0000", 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385693+0000", 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0998999999999999e-05 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385710+0000", 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7280000000000001e-05 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385719+0000", 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7940000000000008e-06 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385785+0000", 2024-04-23T23:50:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6109e-05 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385807+0000", 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-05 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387778+0000", 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019710769999999999 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387822+0000", 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3665000000000003e-05 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1576 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'17, mlcod=29'15)", 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.438628+0000", 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.5806109149999998, 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068440199999999997, 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438628+0000", 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438628+0000", 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438629+0000", 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4589999999999999e-06 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438638+0000", 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0759999999999992e-06 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438656+0000", 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7711999999999999e-05 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438673+0000", 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6939000000000002e-05 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438738+0000", 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5152000000000002e-05 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438761+0000", 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3538000000000001e-05 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.439264+0000", 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050254599999999998 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.439312+0000", 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999998e-05 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1666 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'18, mlcod=29'16)", 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.537430+0000", 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.4818089199999998, 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070912099999999995, 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537430+0000", 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537430+0000", 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537431+0000", 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.409e-06 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537440+0000", 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8610000000000004e-06 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537459+0000", 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8858000000000001e-05 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537469+0000", 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0519999999999999e-05 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537532+0000", 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3155000000000001e-05 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.537554+0000", 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1871000000000001e-05 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.538081+0000", 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052682599999999999 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.538139+0000", 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7621000000000003e-05 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1700 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'19, mlcod=29'17)", 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.4464086109999998, 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090769799999999997, 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572832+0000", 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572856+0000", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4743e-05 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572874+0000", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7666e-05 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572883+0000", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4290000000000007e-06 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572946+0000", 2024-04-23T23:50:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000005e-05 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572980+0000", 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4637000000000001e-05 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.573695+0000", 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071479400000000004 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.573738+0000", 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2466999999999998e-05 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1728 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'20, mlcod=29'18)", 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.4036793869999999, 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022012160000000002, 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615560+0000", 2024-04-23T23:50:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.136e-06 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615580+0000", 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9409e-05 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615597+0000", 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.772e-05 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615606+0000", 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6780000000000006e-06 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615677+0000", 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0714000000000001e-05 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615701+0000", 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4076000000000001e-05 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617717+0000", 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020162729999999999 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617760+0000", 2024-04-23T23:50:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3210000000000001e-05 2024-04-23T23:50:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.098 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:01.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:01.100 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:01.100 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:01.101 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:01.102 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:01.102 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:01.113+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:01.200 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:01.214+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:01.242 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:01.242 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:01.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:01.301 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:01.315+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:01.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:50:01.355 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:01.355 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:01.355 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:01.355 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:01.355 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:01.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:01.401 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:01.415+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:01.420 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone5 from 0xedb95ae (overlap 0xe9a39d1) to 0x175d294 2024-04-23T23:50:01.502 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:01.516+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:01.506 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1133 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.914627+0000", 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6266145979999997, 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044350170000000003, 2024-04-23T23:50:01.530 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1133 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914627+0000", 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914627+0000", 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914630+0000", 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3299999999999999e-06 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914649+0000", 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.88e-05 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914679+0000", 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0429e-05 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914694+0000", 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4545999999999999e-05 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914780+0000", 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300999999999999e-05 2024-04-23T23:50:01.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914871+0000", 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.132e-05 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915301+0000", 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042962600000000001 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918824+0000", 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003523263 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918949+0000", 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012529500000000001 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.919007+0000", 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7853999999999999e-05 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.919062+0000", 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4252999999999997e-05 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1134 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.914736+0000", 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6265051650000002, 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053773800000000002, 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1134 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914736+0000", 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914736+0000", 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914737+0000", 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3799999999999997e-07 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914741+0000", 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8219999999999997e-06 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914757+0000", 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.607e-05 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914769+0000", 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.116e-05 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915872+0000", 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011033200000000001 2024-04-23T23:50:01.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915974+0000", 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101885 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916296+0000", 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032181400000000002 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918866+0000", 2024-04-23T23:50:01.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025704069999999998 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920001+0000", 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011347830000000001 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920057+0000", 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5705999999999998e-05 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920113+0000", 2024-04-23T23:50:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6875e-05 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1135 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.914926+0000", 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6263150030000002, 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052889870000000002, 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1135 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914926+0000", 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914926+0000", 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914927+0000", 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5700000000000002e-07 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914931+0000", 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5920000000000002e-06 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914941+0000", 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1900000000000001e-06 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.914946+0000", 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916577+0000", 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001630982 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916633+0000", 2024-04-23T23:50:01.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6171000000000001e-05 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916851+0000", 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021842300000000001 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918935+0000", 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020834339999999999 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920124+0000", 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011892129999999999 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920174+0000", 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9843999999999999e-05 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920215+0000", 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1408000000000001e-05 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1136 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.915018+0000", 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6262227029999998, 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063018900000000001, 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1136 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915018+0000", 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915018+0000", 2024-04-23T23:50:01.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915019+0000", 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8899999999999998e-07 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915024+0000", 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5279999999999997e-06 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915032+0000", 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0199999999999994e-06 2024-04-23T23:50:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.915037+0000", 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5679999999999999e-06 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.917244+0000", 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002207082 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.917301+0000", 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6588000000000001e-05 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.917655+0000", 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035402399999999998 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920226+0000", 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025707590000000002 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921202+0000", 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097639500000000002 2024-04-23T23:50:01.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921263+0000", 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1268999999999999e-05 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921320+0000", 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6867999999999998e-05 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1138 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.916074+0000", 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6251670540000003, 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054293989999999997, 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1138 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916074+0000", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916074+0000", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916076+0000", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916084+0000", 2024-04-23T23:50:01.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9149999999999996e-06 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916104+0000", 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9545999999999999e-05 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916113+0000", 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0359999999999995e-06 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918072+0000", 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019588689999999998 2024-04-23T23:50:01.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918173+0000", 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101298 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.918503+0000", 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032986000000000002 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921332+0000", 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028294280000000002 2024-04-23T23:50:01.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921398+0000", 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6012000000000004e-05 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921456+0000", 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7988000000000001e-05 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921504+0000", 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7225000000000003e-05 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1139 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.916131+0000", 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6251099880000002, 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013471244, 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1139 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916131+0000", 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916131+0000", 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916132+0000", 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2600000000000002e-07 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916137+0000", 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2170000000000002e-06 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916144+0000", 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0439999999999998e-06 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916150+0000", 2024-04-23T23:50:01.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5339999999999999e-06 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.919100+0000", 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029509250000000001 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.919190+0000", 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9081999999999999e-05 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.919573+0000", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038387499999999999 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922606+0000", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030325389999999999 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929542+0000", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069358900000000001 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929574+0000", 2024-04-23T23:50:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1872000000000002e-05 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929602+0000", 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8739999999999999e-05 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1140 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.916157+0000", 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.625084142, 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013547316, 2024-04-23T23:50:01.565 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1140 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916157+0000", 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916157+0000", 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916157+0000", 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916160+0000", 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7470000000000001e-06 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916166+0000", 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6999999999999996e-06 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916171+0000", 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0089999999999998e-06 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920300+0000", 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004129181 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920400+0000", 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9825000000000001e-05 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.920710+0000", 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030969199999999998 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.925136+0000", 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044261810000000004 2024-04-23T23:50:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929646+0000", 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045106390000000003 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929677+0000", 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0848e-05 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929704+0000", 2024-04-23T23:50:01.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7104e-05 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1141 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.916178+0000", 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6250628379999998, 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013613096, 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1141 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916178+0000", 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916178+0000", 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916179+0000", 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916182+0000", 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.796e-06 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916189+0000", 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9389999999999997e-06 2024-04-23T23:50:01.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.916195+0000", 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921538+0000", 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053425900000000004 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.921636+0000", 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8226000000000004e-05 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922182+0000", 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054546200000000003 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.928752+0000", 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065704939999999996 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929735+0000", 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000982915 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929762+0000", 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7412000000000001e-05 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.929791+0000", 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8957000000000001e-05 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1146 3.8 3:10343aa6:::rbd_data.1179984961fa.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15728640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.922631+0000", 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.618610442, 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062113860000000002, 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1146 2024-04-23T23:50:01.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922631+0000", 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922631+0000", 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922632+0000", 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.593e-06 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922639+0000", 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6270000000000002e-06 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922652+0000", 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3389e-05 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922659+0000", 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922714+0000", 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4305999999999998e-05 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922835+0000", 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121385 2024-04-23T23:50:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.922991+0000", 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015557799999999999 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.925634+0000", 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026432389999999999 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.928767+0000", 2024-04-23T23:50:01.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031328369999999999 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.928814+0000", 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7116000000000001e-05 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.928842+0000", 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8311e-05 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1253 3.8 3:10343aa6:::rbd_data.1179984961fa.0000000000000005:head [stat,truncate 8650752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.079573+0000", 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.461667695, 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054996430000000002, 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1253 2024-04-23T23:50:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079573+0000", 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079573+0000", 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079574+0000", 2024-04-23T23:50:01.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1500000000000004e-07 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079578+0000", 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5099999999999999e-06 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079589+0000", 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0842000000000001e-05 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079595+0000", 2024-04-23T23:50:01.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6769999999999998e-06 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079654+0000", 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9005000000000002e-05 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079714+0000", 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9732999999999999e-05 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-04-23T23:50:01.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.079812+0000", 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7706000000000003e-05 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.080363+0000", 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055155199999999997 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.084983+0000", 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004619761 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.085031+0000", 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7793000000000001e-05 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.085073+0000", 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2348999999999997e-05 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1437 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.295398+0000", 2024-04-23T23:50:01.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.2458435539999999, 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041019510000000004, 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1437 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295398+0000", 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295398+0000", 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295399+0000", 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0359999999999999e-06 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295404+0000", 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1830000000000002e-06 2024-04-23T23:50:01.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295415+0000", 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1367e-05 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295421+0000", 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.896e-06 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295470+0000", 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9101000000000003e-05 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295560+0000", 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0282000000000001e-05 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.295942+0000", 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038171499999999997 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.296681+0000", 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073846700000000003 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.299401+0000", 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002720606 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.299457+0000", 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5559000000000001e-05 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.299500+0000", 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2738999999999999e-05 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.593 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1519 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.382025+0000", 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1592163289999999, 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002915656, 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1519 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382025+0000", 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382025+0000", 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382026+0000", 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5799999999999999e-07 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382031+0000", 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5890000000000002e-06 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382041+0000", 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8519999999999999e-06 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382046+0000", 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9370000000000003e-06 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382084+0000", 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8389000000000001e-05 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382139+0000", 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4625e-05 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.382349+0000", 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021033400000000001 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.384768+0000", 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024189860000000001 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.384884+0000", 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116091 2024-04-23T23:50:01.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.384915+0000", 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0613e-05 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.384941+0000", 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582e-05 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1575 3.8 3:10343aa6:::rbd_data.1179984961fa.0000000000000005:head [truncate 9699328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.437981+0000", 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1032599279999999, 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054512400000000004, 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1575 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.437981+0000", 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.437981+0000", 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.437982+0000", 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3300000000000001e-07 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.437986+0000", 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4880000000000001e-06 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438001+0000", 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4664000000000001e-05 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438009+0000", 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7829999999999999e-06 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438068+0000", 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8643000000000001e-05 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438143+0000", 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5647000000000003e-05 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.438248+0000", 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105148 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.440502+0000", 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002253849 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.443348+0000", 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028453200000000001 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.443384+0000", 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6764999999999998e-05 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.443432+0000", 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8199999999999999e-05 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1655 3.19 e29/22 3:9d0842ad:::rbd_data.1179984961fa.000000000000000b:head v 29'9, mlcod=29'7)", 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.529202+0000", 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.0120396280000001, 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043069629999999996, 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529202+0000", 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529202+0000", 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529202+0000", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529207+0000", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3140000000000003e-06 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529216+0000", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8380000000000008e-06 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529222+0000", 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0190000000000003e-06 2024-04-23T23:50:01.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529286+0000", 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3252999999999999e-05 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.529309+0000", 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3317000000000001e-05 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.533457+0000", 2024-04-23T23:50:01.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004147787 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.533509+0000", 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1796999999999999e-05 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1727 3.8 3:10343aa6:::rbd_data.1179984961fa.0000000000000005:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.614927+0000", 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9263142110000002, 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031712760000000002, 2024-04-23T23:50:01.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1727 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.614927+0000", 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.614927+0000", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.614928+0000", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.105e-06 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.614933+0000", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3449999999999996e-06 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.614949+0000", 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5653000000000001e-05 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.614957+0000", 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5270000000000001e-06 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615023+0000", 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6130000000000006e-05 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615104+0000", 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1577999999999997e-05 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615220+0000", 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011609500000000001 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.616355+0000", 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001134868 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617999+0000", 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016433159999999999 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.618050+0000", 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.185e-05 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.618098+0000", 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7809000000000003e-05 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:1732 3.b 3:d0e475dc:::rbd_object_map.1179984961fa:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.623762+0000", 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9174793239999999, 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040543319999999999, 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1732 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623762+0000", 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623762+0000", 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623763+0000", 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5090000000000001e-06 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623773+0000", 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0029e-05 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623791+0000", 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7856e-05 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623801+0000", 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7389999999999994e-06 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623860+0000", 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8853e-05 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.623926+0000", 2024-04-23T23:50:01.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6335999999999996e-05 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.624103+0000", 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000176825 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.624788+0000", 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068502600000000004 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.627741+0000", 2024-04-23T23:50:01.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029531219999999999 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.627775+0000", 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3442000000000003e-05 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.627816+0000", 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1594999999999997e-05 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.620 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:2724 3.1e 3:7c5de93e:::rbd_header.11798eaf3f6b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.368040+0000", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.17320076100000001, 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063690680000000003, 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 2724 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368040+0000", 2024-04-23T23:50:01.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368040+0000", 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368042+0000", 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.714e-06 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368051+0000", 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-04-23T23:50:01.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368066+0000", 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5187e-05 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368074+0000", 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9060000000000002e-06 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368125+0000", 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1063999999999998e-05 2024-04-23T23:50:01.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.368129+0000", 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4179999999999999e-06 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.372841+0000", 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047111979999999998 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.372900+0000", 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9336999999999998e-05 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.373123+0000", 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000222835 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.373694+0000", 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000570986 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.374348+0000", 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065403499999999997 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.374382+0000", 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4585999999999997e-05 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.374409+0000", 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7010999999999999e-05 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:2864 3.15 e32/22 3:aa3dfb00:::rbd_object_map.11798eaf3f6b:head v 32'24, mlcod=32'22)", 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.450404+0000", 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.090837289000000002, 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033727480000000001, 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450404+0000", 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450404+0000", 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450405+0000", 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3399999999999998e-07 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450411+0000", 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8569999999999997e-06 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450426+0000", 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4251e-05 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450433+0000", 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.075e-06 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450510+0000", 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7622999999999999e-05 2024-04-23T23:50:01.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.450531+0000", 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0911e-05 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.453738+0000", 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032068610000000001 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.453777+0000", 2024-04-23T23:50:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8436000000000003e-05 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3041 3.15 e32/22 3:aa3dfb00:::rbd_object_map.11798eaf3f6b:head v 32'27, mlcod=32'24)", 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459934+0000", 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.08130685, 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032420080000000002, 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459934+0000", 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459934+0000", 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459935+0000", 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2300000000000001e-07 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459941+0000", 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9340000000000003e-06 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459951+0000", 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9699999999999994e-06 2024-04-23T23:50:01.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459957+0000", 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0419999999999999e-06 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460029+0000", 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1905999999999999e-05 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460047+0000", 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8386999999999999e-05 2024-04-23T23:50:01.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.463114+0000", 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030665309999999999 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.463176+0000", 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2614999999999996e-05 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3043 3.15 e32/22 3:aa3dfb00:::rbd_object_map.11798eaf3f6b:head v 32'28, mlcod=32'24)", 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.460810+0000", 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.080430982999999998, 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048860489999999999, 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460810+0000", 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460810+0000", 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460811+0000", 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460816+0000", 2024-04-23T23:50:01.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2789999999999998e-06 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460828+0000", 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2099000000000001e-05 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460836+0000", 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7950000000000008e-06 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460882+0000", 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6366000000000003e-05 2024-04-23T23:50:01.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.460903+0000", 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0692000000000001e-05 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465647+0000", 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047436889999999997 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465696+0000", 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9543999999999999e-05 2024-04-23T23:50:01.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:01.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:01.641 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:01.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:01.643 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:01.656+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:01.743 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:01.756+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:01.748 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:01.748 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:01.748 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:01.748 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:01.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:01.808 INFO:teuthology.orchestra.run.smithi092.stdout:522 read 0x68cde63 thru 0x68d0744 (0x28e2 bytes) 2024-04-23T23:50:01.809 INFO:teuthology.orchestra.run.smithi092.stdout:523 write 0x7c50005 thru 0x7c5839a (0x8396 bytes) 2024-04-23T23:50:01.819 INFO:teuthology.orchestra.run.smithi092.stdout:524 read 0x6e29289 thru 0x6e320ac (0x8e24 bytes) 2024-04-23T23:50:01.820 INFO:teuthology.orchestra.run.smithi092.stdout:525 read 0xb74afa9 thru 0xb74e14b (0x31a3 bytes) 2024-04-23T23:50:01.820 INFO:teuthology.orchestra.run.smithi092.stdout:526 write 0x186bef4 thru 0x1873072 (0x717f bytes) 2024-04-23T23:50:01.825 INFO:teuthology.orchestra.run.smithi092.stdout:528 writesame 0xa68fe92 thru 0xa690a03 data_size 0x1(0xb72 bytes) 2024-04-23T23:50:01.830 INFO:teuthology.orchestra.run.smithi092.stdout:529 write 0x397373a thru 0x397b9ca (0x8291 bytes) 2024-04-23T23:50:01.835 INFO:teuthology.orchestra.run.smithi092.stdout:530 write 0x59871ec thru 0x5995696 (0xe4ab bytes) 2024-04-23T23:50:01.842 INFO:teuthology.orchestra.run.smithi092.stdout:531 write 0xb5b264 thru 0xb5ff1a (0x4cb7 bytes) 2024-04-23T23:50:01.844 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:01.857+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:01.844 INFO:teuthology.orchestra.run.smithi092.stdout:532 write 0x649562f thru 0x64a53e9 (0xfdbb bytes) 2024-04-23T23:50:01.848 INFO:teuthology.orchestra.run.smithi092.stdout:533 read 0xe6a136c thru 0xe6a6691 (0x5326 bytes) 2024-04-23T23:50:01.848 INFO:teuthology.orchestra.run.smithi092.stdout:535 write 0x731e46e thru 0x732a97f (0xc512 bytes) 2024-04-23T23:50:01.852 INFO:teuthology.orchestra.run.smithi092.stdout:536 read 0x295c17f thru 0x296162a (0x54ac bytes) 2024-04-23T23:50:01.854 INFO:teuthology.orchestra.run.smithi092.stdout:538 trunc from 0xedb95ae to 0xd69fad4 2024-04-23T23:50:01.872 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:01.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:01.872 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:01.872 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:01.872 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:01.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:01.883 INFO:teuthology.orchestra.run.smithi092.stdout:539 writesame 0x9f8b46c thru 0x9f8b68d data_size 0x3(0x222 bytes) 2024-04-23T23:50:01.887 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:01.887 INFO:teuthology.orchestra.run.smithi092.stdout:542 writesame 0x58c0ff3 thru 0x58cc42e data_size 0x1e(0xb43c bytes) 2024-04-23T23:50:01.892 INFO:teuthology.orchestra.run.smithi092.stdout:543 read 0x4b6823 thru 0x4b9d45 (0x3523 bytes) 2024-04-23T23:50:01.898 INFO:teuthology.orchestra.run.smithi092.stdout:544 writesame 0xda9427c thru 0xda9c38b data_size 0x3b(0x8110 bytes) 2024-04-23T23:50:01.915 INFO:teuthology.orchestra.run.smithi092.stdout:546 trunc from 0xda9c38c to 0xf6d540 2024-04-23T23:50:01.944 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:01.958+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:01.981 INFO:teuthology.orchestra.run.smithi092.stdout:547 writesame 0xeb64d31 thru 0xeb6b356 data_size 0x19(0x6626 bytes) 2024-04-23T23:50:01.983 INFO:teuthology.orchestra.run.smithi092.stdout:548 trunc from 0xeb6b357 to 0xcad5954 2024-04-23T23:50:01.990 INFO:teuthology.orchestra.run.smithi092.stdout:550 punch from 0x3b7b288 to 0x3b85d31, (0xaaa9 bytes) 2024-04-23T23:50:01.991 INFO:teuthology.orchestra.run.smithi092.stdout:552 punch from 0x3ac8074 to 0x3ad03b4, (0x8340 bytes) 2024-04-23T23:50:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:624 3.1c e28/22 3:3d167183:::rbd_data.1179d7437ec6.0000000000000020:head v 28'6, mlcod=28'4)", 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.985082+0000", 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0256458620000002, 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038804550000000001, 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985082+0000", 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985082+0000", 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985083+0000", 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5499999999999997e-07 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985089+0000", 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0159999999999997e-06 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985099+0000", 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0071000000000001e-05 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985106+0000", 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7059999999999998e-06 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985151+0000", 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5305000000000001e-05 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.985169+0000", 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8119999999999999e-05 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.988933+0000", 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037637830000000001 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.988963+0000", 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9598999999999999e-05 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1143 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.0898322580000004, 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017226057999999999, 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1143 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920898+0000", 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.846e-06 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923748+0000", 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028504649999999999 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923773+0000", 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4311000000000002e-05 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923784+0000", 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1212e-05 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923853+0000", 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9572000000000004e-05 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923965+0000", 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001114 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924349+0000", 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038400400000000002 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937275+0000", 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012926725 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938026+0000", 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075019300000000002 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938068+0000", 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2809999999999998e-05 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938122+0000", 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3520000000000003e-05 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1142 3.1b e29/22 3:dd409f93:::rbd_data.1179984961fa.0000000000000002:head v 29'17, mlcod=29'15)", 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.0866647909999996, 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014083770000000001, 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924064+0000", 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7000000000000003e-07 2024-04-23T23:50:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929770+0000", 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057057829999999999 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929783+0000", 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3118999999999999e-05 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929792+0000", 2024-04-23T23:50:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6549999999999993e-06 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929854+0000", 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2498999999999999e-05 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929871+0000", 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7007000000000002e-05 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938109+0000", 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082374490000000009 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938147+0000", 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8287999999999997e-05 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1148 3.a e29/22 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head v 29'8, mlcod=28'6)", 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.0803459499999999, 2024-04-23T23:50:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067411720000000001, 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930383+0000", 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8999999999999995e-07 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930391+0000", 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7209999999999994e-06 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930402+0000", 2024-04-23T23:50:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0986e-05 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930408+0000", 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.703e-06 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930466+0000", 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7485000000000003e-05 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930485+0000", 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8936e-05 2024-04-23T23:50:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937077+0000", 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065921360000000002 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937123+0000", 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6415000000000002e-05 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1445 3.a e29/22 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head v 29'19, mlcod=29'17)", 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.300286+0000", 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.7104420039999999, 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031696720000000001, 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300286+0000", 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300286+0000", 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300287+0000", 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9500000000000002e-07 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300292+0000", 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2249999999999999e-06 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300302+0000", 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8090000000000004e-06 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300308+0000", 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8799999999999996e-06 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300364+0000", 2024-04-23T23:50:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6755000000000001e-05 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300385+0000", 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0712000000000001e-05 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303422+0000", 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030370789999999998 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303456+0000", 2024-04-23T23:50:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3617e-05 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1579 3.3 3:c01e78db:::rbd_header.1179984961fa:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.443666+0000", 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.567061764, 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030071220000000001, 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1579 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443666+0000", 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443666+0000", 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443668+0000", 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8059999999999999e-06 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443679+0000", 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0499999999999999e-05 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443706+0000", 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7356e-05 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443720+0000", 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3682999999999999e-05 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443834+0000", 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011395099999999999 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.443961+0000", 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012776700000000001 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.444193+0000", 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023129800000000001 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.444606+0000", 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000413688 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.446619+0000", 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002012284 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.446649+0000", 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9873e-05 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.446674+0000", 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4916e-05 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:2812 3.1b e32/22 3:dbf5feff:::rbd_data.11798eaf3f6b.0000000000000009:head v 32'40, mlcod=30'37)", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.437066+0000", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.57366222300000003, 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032373300000000001, 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437066+0000", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437066+0000", 2024-04-23T23:50:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437066+0000", 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8500000000000002e-07 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437079+0000", 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2206e-05 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437089+0000", 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0789e-05 2024-04-23T23:50:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437096+0000", 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6109999999999999e-06 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437166+0000", 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9989999999999999e-05 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.437185+0000", 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8640000000000001e-05 2024-04-23T23:50:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440273+0000", 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030883439999999998 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440303+0000", 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0365000000000002e-05 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:2849 3.1a 3:5ae53de2:::rbd_data.1179b25581a8.0000000000000018:7 [sparse-read 5767168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.440624+0000", 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.57010387299999998, 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002990108, 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2849 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440624+0000", 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440624+0000", 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440625+0000", 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9999999999999998e-07 2024-04-23T23:50:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440627+0000", 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.244e-06 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440636+0000", 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0289999999999994e-06 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440641+0000", 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.443426+0000", 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027848650000000001 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.443507+0000", 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1126999999999998e-05 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.443614+0000", 2024-04-23T23:50:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010697299999999999 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:2850 3.1a 3:5ae53de2:::rbd_data.1179b25581a8.0000000000000018:7 [sparse-read 7340032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.440648+0000", 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.57007968399999998, 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030958270000000002, 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2850 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440648+0000", 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440648+0000", 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440649+0000", 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440652+0000", 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3689999999999998e-06 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440666+0000", 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4228e-05 2024-04-23T23:50:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.440673+0000", 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1670000000000004e-06 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.443640+0000", 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029675130000000002 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.443685+0000", 2024-04-23T23:50:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5167999999999999e-05 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.443744+0000", 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8968999999999997e-05 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3364 3.e e32/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 32'36, mlcod=32'32)", 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.886739+0000", 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.12398930499999999, 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034239869999999999, 2024-04-23T23:50:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886739+0000", 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886739+0000", 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886739+0000", 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4000000000000001e-07 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886745+0000", 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4149999999999998e-06 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886755+0000", 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6760000000000002e-06 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886762+0000", 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4869999999999996e-06 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886806+0000", 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4097999999999997e-05 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.886828+0000", 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2177000000000001e-05 2024-04-23T23:50:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.890124+0000", 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032953489999999999 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.890163+0000", 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9144999999999999e-05 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3365 3.e e32/22 3:7757243f:::rbd_data.1179c62058ea.000000000000002e:head v 32'37, mlcod=32'32)", 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.887570+0000", 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.12315831300000001, 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053120509999999999, 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887570+0000", 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887570+0000", 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887570+0000", 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5799999999999999e-07 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887576+0000", 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4299999999999997e-06 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887584+0000", 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6899999999999992e-06 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887589+0000", 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9089999999999997e-06 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887634+0000", 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4125999999999997e-05 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.887656+0000", 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2371e-05 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.892845+0000", 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051890230000000001 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.892882+0000", 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6943999999999997e-05 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3378 3.11 e32/22 3:8a2d7b3b:::rbd_data.1179c62058ea.000000000000003b:head v 32'1, mlcod=0'0)", 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.888406+0000", 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.122322579, 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044775409999999998, 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888406+0000", 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888406+0000", 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888406+0000", 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2300000000000001e-07 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888418+0000", 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1744e-05 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888426+0000", 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3799999999999994e-06 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888431+0000", 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4989999999999996e-06 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888474+0000", 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3019000000000003e-05 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.888485+0000", 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1625e-05 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.892845+0000", 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043595190000000001 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.892883+0000", 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8232000000000003e-05 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3436 3.1d e32/22 3:b8481f77:::rbd_data.1179c62058ea.0000000000000035:head v 32'4, mlcod=28'2)", 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.936048+0000", 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.074680072, 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043625479999999999, 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936048+0000", 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936048+0000", 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936049+0000", 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1500000000000004e-07 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936054+0000", 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.66e-06 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936065+0000", 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0647e-05 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936080+0000", 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4898e-05 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936145+0000", 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5213000000000001e-05 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936166+0000", 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0749000000000001e-05 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940372+0000", 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042061030000000001 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940411+0000", 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8563000000000003e-05 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3442 3.11 e32/22 3:8a2d7b3b:::rbd_data.1179c62058ea.000000000000003b:head v 32'2, mlcod=0'0)", 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.936567+0000", 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.074160693999999999, 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003847474, 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936567+0000", 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936567+0000", 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936568+0000", 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4799999999999998e-07 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936572+0000", 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.557e-06 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936577+0000", 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6180000000000003e-06 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936581+0000", 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4960000000000001e-06 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936631+0000", 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0343999999999998e-05 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.936654+0000", 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2569999999999999e-05 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940379+0000", 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037258500000000002 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940415+0000", 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5491000000000003e-05 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3544 3.e e32/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 32'48, mlcod=32'45)", 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.945955+0000", 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.064772818999999995, 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038961600000000001, 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.945955+0000", 2024-04-23T23:50:02.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.945955+0000", 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.945956+0000", 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2200000000000004e-07 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.945971+0000", 2024-04-23T23:50:02.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4917e-05 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.945984+0000", 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.945990+0000", 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.888e-06 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946035+0000", 2024-04-23T23:50:02.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4946e-05 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946054+0000", 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9094999999999999e-05 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.949797+0000", 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003743021 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.949851+0000", 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.401e-05 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3545 3.e e32/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 32'49, mlcod=32'45)", 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.946629+0000", 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.064099354999999997, 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033424750000000001, 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946629+0000", 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946629+0000", 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946629+0000", 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1900000000000003e-07 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946634+0000", 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0479999999999998e-06 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946644+0000", 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4259999999999992e-06 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946650+0000", 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1040000000000003e-06 2024-04-23T23:50:02.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946697+0000", 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6777999999999998e-05 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.946716+0000", 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8865e-05 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.949920+0000", 2024-04-23T23:50:02.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032042160000000002 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.949971+0000", 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1518999999999999e-05 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3558 3.a e32/22 3:526f43e1:::rbd_data.1179c62058ea.000000000000000d:head v 32'42, mlcod=32'40)", 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.953585+0000", 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.057142900000000003, 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043091129999999998, 2024-04-23T23:50:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953585+0000", 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953585+0000", 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953586+0000", 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1200000000000002e-07 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953592+0000", 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9399999999999999e-06 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953603+0000", 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1008e-05 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953609+0000", 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2740000000000004e-06 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953683+0000", 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3664999999999994e-05 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953702+0000", 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8882999999999999e-05 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.957836+0000", 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041342189999999997 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.957894+0000", 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8312e-05 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.075 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3555 3.e e32/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 32'56, mlcod=32'52)", 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.953695+0000", 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.057033318999999999, 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042070290000000002, 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953695+0000", 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953695+0000", 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953695+0000", 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2699999999999999e-07 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953700+0000", 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.831e-06 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953711+0000", 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.066e-05 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953717+0000", 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3289999999999999e-06 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953784+0000", 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6302000000000003e-05 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953801+0000", 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7567999999999998e-05 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.957863+0000", 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040613749999999999 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.957902+0000", 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9337e-05 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.079 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3564 3.e e32/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 32'58, mlcod=32'55)", 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.959979+0000", 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.050748994999999998, 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028951160000000001, 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959979+0000", 2024-04-23T23:50:02.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959979+0000", 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959980+0000", 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6700000000000003e-07 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959984+0000", 2024-04-23T23:50:02.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4449999999999998e-06 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959993+0000", 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4810000000000006e-06 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959999+0000", 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.960040+0000", 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1541000000000001e-05 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.960053+0000", 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2758000000000001e-05 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.962821+0000", 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002768132 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.962874+0000", 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3099000000000001e-05 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3565 3.1c e32/22 3:3a1edd37:::rbd_data.1179c62058ea.000000000000000a:head v 32'16, mlcod=31'14)", 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.961077+0000", 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.049650867000000001, 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040952030000000004, 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961077+0000", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961077+0000", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961078+0000", 2024-04-23T23:50:02.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3900000000000005e-07 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961082+0000", 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0640000000000004e-06 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961091+0000", 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8040000000000007e-06 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961095+0000", 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5369999999999999e-06 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961143+0000", 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8108000000000001e-05 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.961165+0000", 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1205000000000001e-05 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.965145+0000", 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039799750000000002 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.965172+0000", 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7971e-05 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.088 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:02.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:02.091 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:02.103+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:02.092 INFO:teuthology.orchestra.run.smithi092.stdout:554 trunc from 0xcad5954 to 0xb232c4 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:557 read 0x7fb84 thru 0x8eb06 (0xef83 bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:558 trunc from 0xb232c4 to 0x16a56a6 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:559 read 0x14acfe thru 0x14f004 (0x4307 bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:560 writesame 0x4aec337 thru 0x4af732b data_size 0x69(0xaff5 bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:564 trunc from 0x4af732c to 0x60a10a7 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:565 write 0xae10892 thru 0xae1b00d (0xa77c bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:568 read 0x7d9192e thru 0x7d9942b (0x7afe bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:570 read 0x1e6b350 thru 0x1e7b0b6 (0xfd67 bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:571 write 0x18597b9 thru 0x1861fae (0x87f6 bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:574 write 0x848beee thru 0x849588c (0x999f bytes) 2024-04-23T23:50:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:579 write 0x79ffcec thru 0x7a038c1 (0x3bd6 bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:580 write 0x4fed4f0 thru 0x4ff7c28 (0xa739 bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:581 read 0x8ce5811 thru 0x8cf1b8c (0xc37c bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:583 read 0x92cda46 thru 0x92d862e (0xabe9 bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:584 read 0x8e29da4 thru 0x8e2fd51 (0x5fae bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:588 punch from 0x93b4234 to 0x93b7d05, (0x3ad1 bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:590 read 0x97deed8 thru 0x97e55a6 (0x66cf bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:591 trunc from 0xae1b00e to 0x9594dc0 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:592 punch from 0x5408e to 0x5864e, (0x45c0 bytes) 2024-04-23T23:50:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:593 read 0x7b6ec46 thru 0x7b7e074 (0xf42f bytes) 2024-04-23T23:50:02.100 INFO:teuthology.orchestra.run.smithi092.stdout:594 writesame 0xddf692a thru 0xddf6a16 data_size 0x3(0xed bytes) 2024-04-23T23:50:02.101 INFO:teuthology.orchestra.run.smithi092.stdout:596 trunc from 0xddf6a17 to 0xe957ba7 2024-04-23T23:50:02.107 INFO:teuthology.orchestra.run.smithi092.stdout:597 read 0x9b2aebb thru 0x9b33592 (0x86d8 bytes) 2024-04-23T23:50:02.107 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.107 INFO:teuthology.orchestra.run.smithi092.stdout:600 read 0x65f83cf thru 0x6600404 (0x8036 bytes) 2024-04-23T23:50:02.108 INFO:teuthology.orchestra.run.smithi092.stdout:601 punch from 0xaaecfcf to 0xaafcce9, (0xfd1a bytes) 2024-04-23T23:50:02.111 INFO:teuthology.orchestra.run.smithi092.stdout:602 trunc from 0xe957ba7 to 0xc5d9d00 2024-04-23T23:50:02.118 INFO:teuthology.orchestra.run.smithi092.stdout:605 write 0x6161bb3 thru 0x616fd90 (0xe1de bytes) 2024-04-23T23:50:02.121 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.121 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.126 INFO:teuthology.orchestra.run.smithi092.stdout:608 trunc from 0xc5d9d00 to 0x887e74a 2024-04-23T23:50:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:610 punch from 0x219989b to 0x219b5b3, (0x1d18 bytes) 2024-04-23T23:50:02.144 INFO:teuthology.orchestra.run.smithi092.stdout:614 write 0xa1c6f54 thru 0xa1cfa78 (0x8b25 bytes) 2024-04-23T23:50:02.147 INFO:teuthology.orchestra.run.smithi092.stdout:615 write 0x3c58074 thru 0x3c628e0 (0xa86d bytes) 2024-04-23T23:50:02.151 INFO:teuthology.orchestra.run.smithi092.stdout:617 read 0x41cd9a8 thru 0x41dd61e (0xfc77 bytes) 2024-04-23T23:50:02.151 INFO:teuthology.orchestra.run.smithi092.stdout:618 writesame 0x96ae147 thru 0x96bd9a6 data_size 0x10(0xf860 bytes) 2024-04-23T23:50:02.155 INFO:teuthology.orchestra.run.smithi092.stdout:619 write 0x5a7c084 thru 0x5a82547 (0x64c4 bytes) 2024-04-23T23:50:02.162 INFO:teuthology.orchestra.run.smithi092.stdout:621 write 0x323000a thru 0x3231821 (0x1818 bytes) 2024-04-23T23:50:02.163 INFO:teuthology.orchestra.run.smithi092.stdout:623 clone 7 order 22 su 65536 sc 10 2024-04-23T23:50:02.191 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:02.205+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:02.234 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:02.234 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:02.234 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:02.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:02.292 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:02.306+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:02.355 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:50:02.388 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:02.389 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:02.389 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:02.389 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:02.389 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:02.392 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:02.406+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:02.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:02.427 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone6 from 0xa1cfa79 (overlap 0xb232c4) to 0x591962 2024-04-23T23:50:02.493 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:02.507+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:02.521 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-04-23T23:50:02.551 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:542 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6699267510000002, 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065200570000000001, 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 542 2024-04-23T23:50:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.58e-07 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893965+0000", 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9280000000000002e-06 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893973+0000", 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2729999999999997e-06 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893987+0000", 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4344e-05 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895263+0000", 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012758610000000001 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895316+0000", 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3724999999999998e-05 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895563+0000", 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024641700000000002 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898014+0000", 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002451103 2024-04-23T23:50:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900408+0000", 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00239419 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900456+0000", 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7837999999999998e-05 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900482+0000", 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5919999999999999e-05 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:543 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6698907280000004, 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065321299999999997, 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 543 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.34e-07 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894000+0000", 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1100000000000001e-06 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894005+0000", 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.126e-06 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894009+0000", 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895875+0000", 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018654260000000001 2024-04-23T23:50:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895926+0000", 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1221000000000003e-05 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896212+0000", 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028643100000000002 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898050+0000", 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001837279 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900488+0000", 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024382330000000002 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900512+0000", 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3818000000000001e-05 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900530+0000", 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8278999999999999e-05 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:544 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6698745749999997, 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006558014, 2024-04-23T23:50:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 544 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894016+0000", 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.393e-06 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894019+0000", 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2559999999999998e-06 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894022+0000", 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8490000000000002e-06 2024-04-23T23:50:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896641+0000", 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026193380000000001 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896701+0000", 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0019000000000003e-05 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896940+0000", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023890900000000001 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898076+0000", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001136242 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900534+0000", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024572869999999998 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900555+0000", 2024-04-23T23:50:02.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1872999999999999e-05 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900572+0000", 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6578999999999998e-05 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:545 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:02.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.669863629, 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091656180000000004, 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 545 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894027+0000", 2024-04-23T23:50:02.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.446e-06 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894030+0000", 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.134e-06 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894033+0000", 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3950000000000001e-06 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.897312+0000", 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003278572 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.897367+0000", 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5566000000000003e-05 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.897694+0000", 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032677100000000001 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898554+0000", 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000859953 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903118+0000", 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045641759999999996 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903158+0000", 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9530000000000003e-05 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903191+0000", 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2854000000000001e-05 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:547 3.1d 3:b88df0d1:::rbd_data.1179d7437ec6.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.898269+0000", 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6656194229999999, 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049549420000000004, 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 547 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898269+0000", 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898269+0000", 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898270+0000", 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6500000000000008e-07 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898275+0000", 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9509999999999997e-06 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898288+0000", 2024-04-23T23:50:02.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3264e-05 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898296+0000", 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1500000000000002e-06 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898335+0000", 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9777999999999998e-05 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898409+0000", 2024-04-23T23:50:02.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3369999999999997e-05 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898505+0000", 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6767000000000004e-05 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.899324+0000", 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081891899999999998 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903145+0000", 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038210779999999999 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903189+0000", 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4022e-05 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903224+0000", 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4678000000000002e-05 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:571 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921259+0000", 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6426297109999997, 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049085170000000003, 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 571 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921259+0000", 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921259+0000", 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921259+0000", 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.89e-07 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921262+0000", 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921266+0000", 2024-04-23T23:50:02.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5179999999999998e-06 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921269+0000", 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.304e-06 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922140+0000", 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00087085500000000002 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922191+0000", 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0831000000000002e-05 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922392+0000", 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020087999999999999 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923577+0000", 2024-04-23T23:50:02.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001185389 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.926125+0000", 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025480189999999999 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.926149+0000", 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3995e-05 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.926167+0000", 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8114e-05 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:572 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921286+0000", 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6426022380000003, 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073967379999999999, 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.588 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 572 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921286+0000", 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921286+0000", 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921287+0000", 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7e-07 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921289+0000", 2024-04-23T23:50:02.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.086e-06 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921293+0000", 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.386e-06 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921297+0000", 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3349999999999998e-06 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922700+0000", 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001403577 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922784+0000", 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418999999999995e-05 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923159+0000", 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037584899999999999 2024-04-23T23:50:02.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924999+0000", 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001839799 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928595+0000", 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035961399999999998 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928644+0000", 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8804000000000002e-05 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928683+0000", 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8973000000000001e-05 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:573 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6425707389999999, 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074509830000000004, 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 573 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.595 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:02.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921320+0000", 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0930000000000001e-06 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921325+0000", 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1389999999999997e-06 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921328+0000", 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0489999999999999e-06 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923614+0000", 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022866890000000002 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923665+0000", 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0661000000000002e-05 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923878+0000", 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021355199999999999 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.926772+0000", 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028935850000000002 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928693+0000", 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001920997 2024-04-23T23:50:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928734+0000", 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1304999999999998e-05 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928769+0000", 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4505999999999998e-05 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:575 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921379+0000", 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6425093010000005, 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074341980000000004, 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 575 2024-04-23T23:50:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921379+0000", 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921379+0000", 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921380+0000", 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4200000000000002e-07 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921381+0000", 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.832e-06 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921385+0000", 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921388+0000", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9110000000000002e-06 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924196+0000", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028076749999999999 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924274+0000", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7886999999999995e-05 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924629+0000", 2024-04-23T23:50:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035520300000000001 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.927147+0000", 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025179619999999999 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928774+0000", 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001626976 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928795+0000", 2024-04-23T23:50:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1112e-05 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928814+0000", 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8731000000000001e-05 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:576 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6424834910000001, 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074581659999999996, 2024-04-23T23:50:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 576 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921407+0000", 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8589999999999999e-06 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921411+0000", 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9380000000000002e-06 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921414+0000", 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0290000000000001e-06 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.925028+0000", 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003613808 2024-04-23T23:50:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.925109+0000", 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0989000000000007e-05 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.925461+0000", 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035196400000000002 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.927186+0000", 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017245610000000001 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928817+0000", 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001631696 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928844+0000", 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6976000000000001e-05 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928863+0000", 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8995999999999999e-05 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:638 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.991276+0000", 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.572612543, 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0098125250000000008, 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 638 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991276+0000", 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991276+0000", 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991277+0000", 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3399999999999999e-07 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991280+0000", 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.281e-06 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991289+0000", 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7790000000000001e-06 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991294+0000", 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5740000000000003e-06 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992754+0000", 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001459498 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992844+0000", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0205999999999997e-05 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.993212+0000", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036836300000000002 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.995097+0000", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018850049999999999 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001033+0000", 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059358240000000001 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001065+0000", 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1511999999999999e-05 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001089+0000", 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3849000000000002e-05 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:639 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.991360+0000", 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.5725281449999997, 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097745739999999994, 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 639 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991360+0000", 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991360+0000", 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991361+0000", 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999991e-07 2024-04-23T23:50:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991374+0000", 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2428e-05 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991385+0000", 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0694e-05 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991392+0000", 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.002e-06 2024-04-23T23:50:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.993743+0000", 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023511000000000001 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.993792+0000", 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9607999999999997e-05 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994000+0000", 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000207846 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.995962+0000", 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019620050000000002 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001095+0000", 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051327389999999999 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001118+0000", 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3091e-05 2024-04-23T23:50:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001135+0000", 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7101000000000001e-05 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:640 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.992430+0000", 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.5714584580000004, 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087474370000000003, 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 640 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992430+0000", 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992430+0000", 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992431+0000", 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.012e-06 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992437+0000", 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3759999999999999e-06 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992449+0000", 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2408999999999999e-05 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992457+0000", 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1799999999999996e-06 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994312+0000", 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018544049999999999 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994391+0000", 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9258999999999994e-05 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994748+0000", 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035705200000000001 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.996175+0000", 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00142716 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001139+0000", 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049635410000000001 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001160+0000", 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1676e-05 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001178+0000", 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7367000000000002e-05 2024-04-23T23:50:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1145 3.17 e29/22 3:ef88a9a9:::rbd_data.1179984961fa.0000000000000001:head v 29'9, mlcod=29'7)", 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.924079+0000", 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.639809531, 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013902618, 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924079+0000", 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924079+0000", 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924081+0000", 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.513e-06 2024-04-23T23:50:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928877+0000", 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047967950000000004 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928895+0000", 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7642000000000002e-05 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928904+0000", 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1870000000000003e-06 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928976+0000", 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1870000000000001e-05 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928995+0000", 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8915000000000001e-05 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937940+0000", 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089449739999999996 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937982+0000", 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1721999999999998e-05 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1149 3.17 e29/22 3:e91d246d:::rbd_data.1179984961fa.0000000000000008:head v 29'10, mlcod=29'7)", 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.930374+0000", 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6335146839999997, 2024-04-23T23:50:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076889610000000002, 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930374+0000", 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930374+0000", 2024-04-23T23:50:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930375+0000", 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000001e-07 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4279999999999999e-06 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930396+0000", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3487e-05 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930404+0000", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.931651+0000", 2024-04-23T23:50:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001247492 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.931671+0000", 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0290999999999999e-05 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938028+0000", 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006357003 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938063+0000", 2024-04-23T23:50:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.447e-05 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:2859 3.16 3:695e1318:::rbd_data.11798eaf3f6b.0000000000000006:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.442131+0000", 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.1217573160000001, 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050780030000000002, 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2859 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.442131+0000", 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.442131+0000", 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.442132+0000", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5200000000000002e-07 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.442134+0000", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.855e-06 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.442137+0000", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.895e-06 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.442141+0000", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1860000000000001e-06 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.444095+0000", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001954304 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.444174+0000", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9382000000000007e-05 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.444361+0000", 2024-04-23T23:50:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018665700000000001 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.446972+0000", 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002610696 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.447117+0000", 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014558600000000001 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.447169+0000", 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1382999999999999e-05 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.447209+0000", 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0606999999999999e-05 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3436 3.1d 3:b8481f77:::rbd_data.1179c62058ea.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.935334+0000", 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.62855457199999998, 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005238581, 2024-04-23T23:50:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3436 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935334+0000", 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935334+0000", 2024-04-23T23:50:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935335+0000", 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4300000000000001e-06 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935340+0000", 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0089999999999998e-06 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935355+0000", 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4728e-05 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935363+0000", 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8790000000000003e-06 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935446+0000", 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3150000000000002e-05 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935521+0000", 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4578000000000003e-05 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935975+0000", 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000454673 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940473+0000", 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044979110000000003 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940511+0000", 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7521000000000002e-05 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940541+0000", 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0219999999999999e-05 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940573+0000", 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1482000000000001e-05 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3596 3.1d 3:bafdd96b:::rbd_data.1179c62058ea.0000000000000002:head [stat,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.019892+0000", 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.54399699599999995, 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051972729999999997, 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3596 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019892+0000", 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019892+0000", 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019893+0000", 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6169999999999999e-06 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019901+0000", 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4730000000000003e-06 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019925+0000", 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4632000000000001e-05 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019935+0000", 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7839999999999998e-06 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020013+0000", 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8333999999999999e-05 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020108+0000", 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4852000000000006e-05 2024-04-23T23:50:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020251+0000", 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000142922 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.021103+0000", 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085139899999999997 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025002+0000", 2024-04-23T23:50:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038996510000000001 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025051+0000", 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8349000000000001e-05 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025089+0000", 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8260000000000003e-05 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3604 3.1c 3:3a1edd37:::rbd_data.1179c62058ea.000000000000000a:head [stat,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.020035+0000", 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.54385362299999995, 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059299970000000002, 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3604 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020035+0000", 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020035+0000", 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020036+0000", 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3799999999999997e-07 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020039+0000", 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8070000000000001e-06 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020048+0000", 2024-04-23T23:50:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8340000000000005e-06 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020055+0000", 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3990000000000001e-06 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020114+0000", 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9358000000000003e-05 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020195+0000", 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0679000000000005e-05 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020323+0000", 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012818899999999999 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.021120+0000", 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00079704000000000005 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025908+0000", 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047881369999999996 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025939+0000", 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0539e-05 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025965+0000", 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6477000000000001e-05 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3870 3.1d 3:bafdd96b:::rbd_data.1179c62058ea.0000000000000002:head [stat,truncate 524288] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.464629+0000", 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.099259193999999995, 2024-04-23T23:50:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068574070000000003, 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3870 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464629+0000", 2024-04-23T23:50:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464629+0000", 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464631+0000", 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8109999999999999e-06 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464638+0000", 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5300000000000002e-06 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464652+0000", 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4199000000000001e-05 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464669+0000", 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7070000000000001e-05 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464733+0000", 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3765999999999999e-05 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464831+0000", 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8249000000000001e-05 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.465019+0000", 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018749399999999999 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.466959+0000", 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019400839999999999 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471382+0000", 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044238610000000003 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471439+0000", 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6907000000000003e-05 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471487+0000", 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7435999999999998e-05 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.669 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:02.671 INFO:teuthology.orchestra.run.smithi092.stdout:624 trunc from 0xa1cfa79 to 0xaed25c0 2024-04-23T23:50:02.671 INFO:teuthology.orchestra.run.smithi092.stdout:626 punch from 0x572081d to 0x572d4e5, (0xccc8 bytes) 2024-04-23T23:50:02.671 INFO:teuthology.orchestra.run.smithi092.stdout:628 writesame 0x84fc9c8 thru 0x85087a6 data_size 0x1(0xbddf bytes) 2024-04-23T23:50:02.671 INFO:teuthology.orchestra.run.smithi092.stdout:629 trunc from 0xaed25c0 to 0xe2106dc 2024-04-23T23:50:02.671 INFO:teuthology.orchestra.run.smithi092.stdout:632 trunc from 0xe2106dc to 0x2bc01ff 2024-04-23T23:50:02.671 INFO:teuthology.orchestra.run.smithi092.stdout:633 write 0x137c8a5 thru 0x13863bd (0x9b19 bytes) 2024-04-23T23:50:02.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:02.672 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:02.684+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:02.690 INFO:teuthology.orchestra.run.smithi092.stdout:634 write 0xd574eba thru 0xd57d9d6 (0x8b1d bytes) 2024-04-23T23:50:02.694 INFO:teuthology.orchestra.run.smithi092.stdout:635 punch from 0x926a931 to 0x92755c1, (0xac90 bytes) 2024-04-23T23:50:02.694 INFO:teuthology.orchestra.run.smithi092.stdout:636 punch from 0x930eaea to 0x930eb59, (0x6f bytes) 2024-04-23T23:50:02.694 INFO:teuthology.orchestra.run.smithi092.stdout:638 write 0x2bb86c9 thru 0x2bc5d0b (0xd643 bytes) 2024-04-23T23:50:02.700 INFO:teuthology.orchestra.run.smithi092.stdout:639 write 0xd6e7884 thru 0xd6ec74b (0x4ec8 bytes) 2024-04-23T23:50:02.702 INFO:teuthology.orchestra.run.smithi092.stdout:640 write 0x92baffa thru 0x92c15df (0x65e6 bytes) 2024-04-23T23:50:02.713 INFO:teuthology.orchestra.run.smithi092.stdout:641 write 0xeb2b7ac thru 0xeb2bfc7 (0x81c bytes) 2024-04-23T23:50:02.716 INFO:teuthology.orchestra.run.smithi092.stdout:642 write 0x78bc014 thru 0x78bf8c2 (0x38af bytes) 2024-04-23T23:50:02.718 INFO:teuthology.orchestra.run.smithi092.stdout:644 writesame 0x22273bf thru 0x2230892 data_size 0x4b(0x94d4 bytes) 2024-04-23T23:50:02.727 INFO:teuthology.orchestra.run.smithi092.stdout:645 write 0xc38885a thru 0xc38d457 (0x4bfe bytes) 2024-04-23T23:50:02.733 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.733 INFO:teuthology.orchestra.run.smithi092.stdout:648 read 0x92b57a9 thru 0x92c188e (0xc0e6 bytes) 2024-04-23T23:50:02.733 INFO:teuthology.orchestra.run.smithi092.stdout:649 read 0x4dfc0ab thru 0x4e047ec (0x8742 bytes) 2024-04-23T23:50:02.734 INFO:teuthology.orchestra.run.smithi092.stdout:651 write 0x9cdffd8 thru 0x9ce6abf (0x6ae8 bytes) 2024-04-23T23:50:02.737 INFO:teuthology.orchestra.run.smithi092.stdout:652 write 0x4d10992 thru 0x4d13fbd (0x362c bytes) 2024-04-23T23:50:02.739 INFO:teuthology.orchestra.run.smithi092.stdout:653 write 0xb4c5a2c thru 0xb4cf1aa (0x977f bytes) 2024-04-23T23:50:02.747 INFO:teuthology.orchestra.run.smithi092.stdout:654 trunc from 0xeb2bfc8 to 0xa1c219c 2024-04-23T23:50:02.763 INFO:teuthology.orchestra.run.smithi092.stdout:655 punch from 0x68a09b3 to 0x68a5173, (0x47c0 bytes) 2024-04-23T23:50:02.763 INFO:teuthology.orchestra.run.smithi092.stdout:656 writesame 0x68528d6 thru 0x685fe81 data_size 0x32(0xd5ac bytes) 2024-04-23T23:50:02.766 INFO:teuthology.orchestra.run.smithi092.stdout:657 punch from 0x542f552 to 0x543dfa4, (0xea52 bytes) 2024-04-23T23:50:02.771 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:02.785+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:02.777 INFO:teuthology.orchestra.run.smithi092.stdout:658 writesame 0xc7be70b thru 0xc7cbd12 data_size 0x24(0xd608 bytes) 2024-04-23T23:50:02.777 INFO:teuthology.orchestra.run.smithi092.stdout:659 read 0x89fe78b thru 0x8a0856a (0x9de0 bytes) 2024-04-23T23:50:02.778 INFO:teuthology.orchestra.run.smithi092.stdout:660 trunc from 0xc7cbd13 to 0xcf4d1a7 2024-04-23T23:50:02.784 INFO:teuthology.orchestra.run.smithi092.stdout:662 write 0x450c81c thru 0x4512a67 (0x624c bytes) 2024-04-23T23:50:02.789 INFO:teuthology.orchestra.run.smithi092.stdout:663 write 0x7efe8c1 thru 0x7f058e3 (0x7023 bytes) 2024-04-23T23:50:02.792 INFO:teuthology.orchestra.run.smithi092.stdout:664 write 0x6653c37 thru 0x665881c (0x4be6 bytes) 2024-04-23T23:50:02.794 INFO:teuthology.orchestra.run.smithi092.stdout:665 punch from 0x988010e to 0x98881b5, (0x80a7 bytes) 2024-04-23T23:50:02.794 INFO:teuthology.orchestra.run.smithi092.stdout:669 read 0x840e939 thru 0x8416fc7 (0x868f bytes) 2024-04-23T23:50:02.809 INFO:teuthology.orchestra.run.smithi092.stdout:671 trunc from 0xcf4d1a7 to 0x9878e2 2024-04-23T23:50:02.823 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5200 3.a 3.87c2f64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.836954+0000", 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00093079199999999995, 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00095282500000000003, 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5200 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836954+0000", 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836954+0000", 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836954+0000", 2024-04-23T23:50:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2699999999999999e-07 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836958+0000", 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6289999999999999e-06 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836964+0000", 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6699999999999997e-06 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836970+0000", 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4240000000000001e-06 2024-04-23T23:50:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837910+0000", 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00094060200000000002 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5201 3.a 3.87c2f64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.836977+0000", 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00090722899999999996, 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099923099999999995, 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5201 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836977+0000", 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836977+0000", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836978+0000", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0600000000000001e-07 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836980+0000", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1009999999999999e-06 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836986+0000", 2024-04-23T23:50:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836992+0000", 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3900000000000001e-06 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5202 3.a 3.87c2f64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.836998+0000", 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00088654499999999998, 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010370449999999999, 2024-04-23T23:50:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5202 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836998+0000", 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836998+0000", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.836998+0000", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5200000000000002e-07 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837000+0000", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9300000000000002e-06 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837007+0000", 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4200000000000004e-06 2024-04-23T23:50:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837012+0000", 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5203 3.a 3.87c2f64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.837018+0000", 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00086627299999999998, 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001072929, 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5203 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837018+0000", 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837018+0000", 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837019+0000", 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837021+0000", 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9920000000000002e-06 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837027+0000", 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837032+0000", 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8369999999999996e-06 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5204 3.a 3.87c2f64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.837039+0000", 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.000845899, 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011087580000000001, 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5204 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837039+0000", 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837039+0000", 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837039+0000", 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4199999999999996e-07 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837042+0000", 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4180000000000001e-06 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837049+0000", 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.601e-06 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.837054+0000", 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0520000000000001e-06 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-23T23:50:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 5 2024-04-23T23:50:02.840 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:02.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:02.864 INFO:teuthology.orchestra.run.smithi092.stdout:672 writesame 0xbd30afd thru 0xbd32a15 data_size 0x1(0x1f19 bytes) 2024-04-23T23:50:02.864 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.872 INFO:teuthology.orchestra.run.smithi092.stdout:675 write 0xe1e80d0 thru 0xe1e893f (0x870 bytes) 2024-04-23T23:50:02.872 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:02.886+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:02.874 INFO:teuthology.orchestra.run.smithi092.stdout:676 write 0x4123dc9 thru 0x4130c0d (0xce45 bytes) 2024-04-23T23:50:02.880 INFO:teuthology.orchestra.run.smithi092.stdout:677 writesame 0x92bdb33 thru 0x92c5fad data_size 0x23(0x847b bytes) 2024-04-23T23:50:02.885 INFO:teuthology.orchestra.run.smithi092.stdout:678 write 0x5fda38d thru 0x5fdbae2 (0x1756 bytes) 2024-04-23T23:50:02.889 INFO:teuthology.orchestra.run.smithi092.stdout:679 write 0x7731697 thru 0x773f197 (0xdb01 bytes) 2024-04-23T23:50:02.894 INFO:teuthology.orchestra.run.smithi092.stdout:680 read 0x44fb58d thru 0x450b156 (0xfbca bytes) 2024-04-23T23:50:02.894 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.894 INFO:teuthology.orchestra.run.smithi092.stdout:682 read 0x1859c93 thru 0x185ecb4 (0x5022 bytes) 2024-04-23T23:50:02.901 INFO:teuthology.orchestra.run.smithi092.stdout:683 write 0xe97e3e1 thru 0xe9850a3 (0x6cc3 bytes) 2024-04-23T23:50:02.906 INFO:teuthology.orchestra.run.smithi092.stdout:685 read 0xe06cda2 thru 0xe073fd4 (0x7233 bytes) 2024-04-23T23:50:02.907 INFO:teuthology.orchestra.run.smithi092.stdout:686 punch from 0x8ac01e5 to 0x8accf67, (0xcd82 bytes) 2024-04-23T23:50:02.907 INFO:teuthology.orchestra.run.smithi092.stdout:688 writesame 0x3ee14bd thru 0x3ee1c41 data_size 0x23(0x785 bytes) 2024-04-23T23:50:02.926 INFO:teuthology.orchestra.run.smithi092.stdout:690 trunc from 0xe9850a4 to 0x1bd245d 2024-04-23T23:50:02.950 INFO:teuthology.orchestra.run.smithi092.stdout:691 trunc from 0x1bd245d to 0xdafd47 2024-04-23T23:50:02.950 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:02.951 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:02.951 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:02.951 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:02.951 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:02.959 INFO:teuthology.orchestra.run.smithi092.stdout:693 read 0x741a5 thru 0x809be (0xc81a bytes) 2024-04-23T23:50:02.959 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.959 INFO:teuthology.orchestra.run.smithi092.stdout:697 punch from 0x1d85 to 0xb4d0, (0x974b bytes) 2024-04-23T23:50:02.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:02.974 INFO:teuthology.orchestra.run.smithi092.stdout:698 write 0x861e93c thru 0x8626a41 (0x8106 bytes) 2024-04-23T23:50:02.977 INFO:teuthology.orchestra.run.smithi092.stdout:699 write 0x13e0aec thru 0x13e4081 (0x3596 bytes) 2024-04-23T23:50:02.979 INFO:teuthology.orchestra.run.smithi092.stdout:701 read 0xab2a1e thru 0xab4bf1 (0x21d4 bytes) 2024-04-23T23:50:02.979 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:02.980 INFO:teuthology.orchestra.run.smithi092.stdout:704 punch from 0x51d207e to 0x51dad39, (0x8cbb bytes) 2024-04-23T23:50:02.982 INFO:teuthology.orchestra.run.smithi092.stdout:705 trunc from 0x8626a42 to 0xae2e703 2024-04-23T23:50:02.999 INFO:teuthology.orchestra.run.smithi092.stdout:707 trunc from 0xae2e703 to 0x112bf5a 2024-04-23T23:50:03.005 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:03.019+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:03.034 INFO:teuthology.orchestra.run.smithi092.stdout:710 write 0xcb49270 thru 0xcb5390b (0xa69c bytes) 2024-04-23T23:50:03.048 INFO:teuthology.orchestra.run.smithi092.stdout:711 write 0xcb91f2c thru 0xcb9e923 (0xc9f8 bytes) 2024-04-23T23:50:03.050 INFO:teuthology.orchestra.run.smithi092.stdout:712 read 0xc5b28ac thru 0xc5c2177 (0xf8cc bytes) 2024-04-23T23:50:03.051 INFO:teuthology.orchestra.run.smithi092.stdout:714 read 0x6c8146 thru 0x6d7e99 (0xfd54 bytes) 2024-04-23T23:50:03.051 INFO:teuthology.orchestra.run.smithi092.stdout:715 writesame 0x15e01c3 thru 0x15e72b1 data_size 0x17(0x70ef bytes) 2024-04-23T23:50:03.053 INFO:teuthology.orchestra.run.smithi092.stdout:716 read 0xf78349 thru 0xf86e04 (0xeabc bytes) 2024-04-23T23:50:03.060 INFO:teuthology.orchestra.run.smithi092.stdout:718 write 0xd0d96fe thru 0xd0e0c79 (0x757c bytes) 2024-04-23T23:50:03.065 INFO:teuthology.orchestra.run.smithi092.stdout:720 writesame 0x39a978b thru 0x39b3d79 data_size 0x1(0xa5ef bytes) 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1148 3.a 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15728640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.929815+0000", 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1530407589999996, 2024-04-23T23:50:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075606140000000002, 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1148 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929815+0000", 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929815+0000", 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929817+0000", 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1600000000000001e-06 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929832+0000", 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5034e-05 2024-04-23T23:50:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929854+0000", 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1345999999999999e-05 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929863+0000", 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3209999999999994e-06 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929926+0000", 2024-04-23T23:50:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3570000000000003e-05 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930074+0000", 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014782599999999999 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930283+0000", 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020858899999999999 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.933376+0000", 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003093284 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937270+0000", 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038939000000000001 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937333+0000", 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3177999999999997e-05 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937376+0000", 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2406e-05 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1189 3.a 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head [stat,write 166438~62691] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.996541+0000", 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.086315098, 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030029100000000001, 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1189 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.996541+0000", 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.996541+0000", 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.996543+0000", 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1009999999999999e-06 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.998255+0000", 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001712152 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.998271+0000", 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6072000000000001e-05 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.998285+0000", 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4463e-05 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.998345+0000", 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9623000000000002e-05 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.998416+0000", 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1353999999999995e-05 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.998520+0000", 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103545 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.999345+0000", 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082550600000000005 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.999471+0000", 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012560699999999999 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.999507+0000", 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5679999999999997e-05 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.999544+0000", 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6807000000000002e-05 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1445 3.a 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head [truncate 6553600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.299738+0000", 2024-04-23T23:50:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7831177040000004, 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039146060000000002, 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1445 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299738+0000", 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299738+0000", 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299740+0000", 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.471e-06 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299746+0000", 2024-04-23T23:50:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9970000000000004e-06 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299760+0000", 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4100000000000001e-05 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299768+0000", 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299846+0000", 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7787999999999995e-05 2024-04-23T23:50:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299934+0000", 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8480999999999995e-05 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300062+0000", 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012774100000000001 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300874+0000", 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081191300000000004 2024-04-23T23:50:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303572+0000", 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002698776 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303623+0000", 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0408000000000003e-05 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303653+0000", 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9782e-05 2024-04-23T23:50:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3312 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870589+0000", 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.2122672409999999, 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029578370000000001, 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3312 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870589+0000", 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870589+0000", 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870589+0000", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.37e-07 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870592+0000", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870604+0000", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2119999999999999e-05 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870612+0000", 2024-04-23T23:50:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3909999999999998e-06 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873372+0000", 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027592720000000001 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873453+0000", 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1156999999999996e-05 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873546+0000", 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.357e-05 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3317 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870625+0000", 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.2122307329999999, 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031321589999999998, 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3317 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870625+0000", 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870625+0000", 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870626+0000", 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3499999999999996e-07 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870629+0000", 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7629999999999998e-06 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870651+0000", 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2161999999999999e-05 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870657+0000", 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9399999999999999e-06 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873586+0000", 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029287580000000001 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873676+0000", 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0241999999999994e-05 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873757+0000", 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0759000000000004e-05 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3319 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870668+0000", 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.212187763, 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033255870000000001, 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3319 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870668+0000", 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870668+0000", 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870669+0000", 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5300000000000004e-07 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870672+0000", 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870678+0000", 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.421e-06 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870683+0000", 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6949999999999999e-06 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873811+0000", 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031278859999999999 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873901+0000", 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9771000000000005e-05 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.873994+0000", 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2915000000000001e-05 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3320 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870689+0000", 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.2121667970000001, 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003535608, 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3320 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870689+0000", 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870689+0000", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870689+0000", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8099999999999998e-07 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870691+0000", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9290000000000001e-06 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870697+0000", 2024-04-23T23:50:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.874e-06 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870703+0000", 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4330000000000003e-06 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874026+0000", 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003323524 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874120+0000", 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4135999999999995e-05 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874225+0000", 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104331 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3322 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 16252928~60550] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870711+0000", 2024-04-23T23:50:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.212144289, 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042916940000000004, 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3322 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870711+0000", 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870711+0000", 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870712+0000", 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3500000000000002e-07 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870714+0000", 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.08e-06 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870731+0000", 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7337e-05 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870737+0000", 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874270+0000", 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003532898 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874359+0000", 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8819999999999993e-05 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875003+0000", 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00064445699999999997 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3343 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870988+0000", 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.2118673470000001, 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042296779999999997, 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3343 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870988+0000", 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870988+0000", 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870989+0000", 2024-04-23T23:50:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9100000000000003e-07 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870993+0000", 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.092e-06 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871003+0000", 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8470000000000008e-06 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871010+0000", 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3590000000000002e-06 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875037+0000", 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040269069999999997 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875118+0000", 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0422999999999999e-05 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875218+0000", 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100259 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3344 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.211834098, 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044300880000000004, 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3344 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3499999999999996e-07 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871025+0000", 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3199999999999998e-06 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871031+0000", 2024-04-23T23:50:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4760000000000003e-06 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871036+0000", 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6530000000000002e-06 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875268+0000", 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042325890000000001 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875356+0000", 2024-04-23T23:50:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8016999999999994e-05 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875452+0000", 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5497999999999999e-05 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3345 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.2118099920000001, 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004594992, 2024-04-23T23:50:03.116 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3345 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871048+0000", 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.995e-06 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871054+0000", 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4890000000000003e-06 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871059+0000", 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8799999999999999e-06 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875479+0000", 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044206230000000003 2024-04-23T23:50:03.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875551+0000", 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2089999999999996e-05 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875641+0000", 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9491000000000001e-05 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3346 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.871261+0000", 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.21159463, 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045777150000000004, 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3346 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871261+0000", 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871261+0000", 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871262+0000", 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9299999999999997e-07 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871266+0000", 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5560000000000001e-06 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871278+0000", 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1588e-05 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871286+0000", 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3019999999999995e-06 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875664+0000", 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043780820000000002 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875735+0000", 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0992000000000001e-05 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875839+0000", 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103402 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3426 3.a 3:50bb8fc8:::rbd_data.1179c62058ea.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.932417+0000", 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.150438326, 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034472489999999999, 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3426 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932417+0000", 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932417+0000", 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932419+0000", 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0260000000000002e-06 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932427+0000", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6599999999999995e-06 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932449+0000", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1671999999999999e-05 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932459+0000", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0212e-05 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932530+0000", 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1425000000000001e-05 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932792+0000", 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026129099999999998 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.932908+0000", 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116657 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.934699+0000", 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001790345 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935750+0000", 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010509250000000001 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935805+0000", 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5006999999999997e-05 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935865+0000", 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0028999999999997e-05 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3558 3.a 3:526f43e1:::rbd_data.1179c62058ea.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.953052+0000", 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.1298039499999999, 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050928459999999998, 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.130 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3558 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953052+0000", 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953052+0000", 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953054+0000", 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9910000000000001e-06 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953064+0000", 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0662999999999999e-05 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953081+0000", 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6381000000000001e-05 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953090+0000", 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3840000000000003e-06 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953159+0000", 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8855000000000004e-05 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953306+0000", 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000146775 2024-04-23T23:50:03.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953505+0000", 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000198746 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.955142+0000", 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001637709 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.958042+0000", 2024-04-23T23:50:03.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029001209999999999 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.958102+0000", 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9308000000000002e-05 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.958145+0000", 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2913000000000001e-05 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3607 3.a 3:526f43e1:::rbd_data.1179c62058ea.000000000000000d:head [stat,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.019920+0000", 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0629352480000001, 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052255180000000002, 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3607 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019920+0000", 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019920+0000", 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019922+0000", 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7120000000000001e-06 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019930+0000", 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3189999999999997e-06 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019953+0000", 2024-04-23T23:50:03.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019962+0000", 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5929999999999995e-06 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020033+0000", 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0352999999999995e-05 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020116+0000", 2024-04-23T23:50:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3163000000000003e-05 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020257+0000", 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014111500000000001 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.021149+0000", 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000891931 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025038+0000", 2024-04-23T23:50:03.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038888759999999999 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025094+0000", 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5826999999999999e-05 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025146+0000", 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2410000000000001e-05 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3849 3.a 3:50bb8fc8:::rbd_data.1179c62058ea.0000000000000022:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.444425+0000", 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.63843024999999998, 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046949089999999997, 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3849 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444425+0000", 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444425+0000", 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444426+0000", 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3900000000000003e-07 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444431+0000", 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1799999999999998e-06 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444445+0000", 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4419999999999999e-05 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444452+0000", 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4329999999999998e-06 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444516+0000", 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3076000000000004e-05 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444600+0000", 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4148999999999997e-05 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444769+0000", 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.445664+0000", 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00089460600000000005 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449052+0000", 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003388622 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449082+0000", 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9904999999999999e-05 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449120+0000", 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8071000000000002e-05 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3850 3.10 3:09e2f0c1:::rbd_data.1179c62058ea.0000000000000026:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.444472+0000", 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.63838355199999997, 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045341360000000002, 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3850 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444472+0000", 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444472+0000", 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444473+0000", 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7199999999999999e-07 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444476+0000", 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.875e-06 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444486+0000", 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-05 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444493+0000", 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6329999999999999e-06 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444540+0000", 2024-04-23T23:50:03.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7089000000000003e-05 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444623+0000", 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3634999999999995e-05 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444790+0000", 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000166489 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.446121+0000", 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013310220000000001 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.448903+0000", 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027820810000000001 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.448956+0000", 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2580000000000001e-05 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449006+0000", 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0729999999999997e-05 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:4857 3.a 3:57619caa:::rbd_data.1179c4f11774.000000000000000f:head [stat,write 3014656~10856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.799689+0000", 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.28316625899999998, 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039895010000000003, 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4857 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799689+0000", 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799689+0000", 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799691+0000", 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5680000000000001e-06 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799709+0000", 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8121000000000001e-05 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799725+0000", 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5784e-05 2024-04-23T23:50:03.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799733+0000", 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3180000000000007e-06 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799789+0000", 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.613e-05 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799853+0000", 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3596000000000006e-05 2024-04-23T23:50:03.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799940+0000", 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6590999999999998e-05 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.800680+0000", 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074060300000000005 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.803623+0000", 2024-04-23T23:50:03.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029428060000000001 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.803653+0000", 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0443999999999999e-05 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.803679+0000", 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5539999999999999e-05 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:4870 3.a 3:57619caa:::rbd_data.1179c4f11774.000000000000000f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.826399+0000", 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.25645704800000002, 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030174580000000002, 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4870 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826399+0000", 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826399+0000", 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826400+0000", 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.28e-06 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826406+0000", 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9120000000000003e-06 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826422+0000", 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5877000000000001e-05 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826432+0000", 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9979999999999998e-06 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826498+0000", 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6713999999999999e-05 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826579+0000", 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0592000000000004e-05 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.826675+0000", 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6203e-05 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.827584+0000", 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00090882500000000004 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.829333+0000", 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017491620000000001 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.829376+0000", 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2896999999999999e-05 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.829416+0000", 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9997999999999999e-05 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.164 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5348 3.a 3:51876ddd:::rbd_data.1179c4f11774.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.945890+0000", 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.136966214, 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005120655, 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5348 2024-04-23T23:50:03.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945890+0000", 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945890+0000", 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945891+0000", 2024-04-23T23:50:03.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.615e-06 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945897+0000", 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945913+0000", 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5316e-05 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945921+0000", 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7630000000000006e-06 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945993+0000", 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1143000000000006e-05 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946000+0000", 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.396e-06 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947587+0000", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015866999999999999 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947692+0000", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105004 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947809+0000", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117612 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:03.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948974+0000", 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011645150000000001 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950952+0000", 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019787030000000001 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950985+0000", 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2119000000000002e-05 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:03.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.951010+0000", 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5605e-05 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:03.171 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:03.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:03.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:03.174 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:03.186+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:721 read 0x811b04c thru 0x81207f5 (0x57aa bytes) 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:723 punch from 0x5a961f9 to 0x5aa5b98, (0xf99f bytes) 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:724 punch from 0xa338a6b to 0xa343a0d, (0xafa2 bytes) 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:725 read 0xacaad0f thru 0xacad7c5 (0x2ab7 bytes) 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:726 write 0x3ff445d thru 0x3ff5f9e (0x1b42 bytes) 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:727 write 0xc583581 thru 0xc59051c (0xcf9c bytes) 2024-04-23T23:50:03.176 INFO:teuthology.orchestra.run.smithi092.stdout:730 read 0x441e2be thru 0x442b57c (0xd2bf bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:731 writesame 0xb082f1a thru 0xb08b0ad data_size 0x2(0x8194 bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:732 trunc from 0xd0e0c7a to 0x50cfc2f 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:733 read 0x38d194b thru 0x38e0686 (0xed3c bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:735 trunc from 0x50cfc2f to 0x9882ed9 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:736 write 0x44e7ba6 thru 0x44e9753 (0x1bae bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:737 write 0x8128d1 thru 0x8212cb (0xe9fb bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:739 write 0x396ffd3 thru 0x397f38a (0xf3b8 bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:740 read 0x14bf7f8 thru 0x14c27c0 (0x2fc9 bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:741 read 0x50e3cef thru 0x50ee3e0 (0xa6f2 bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:743 punch from 0x2190722 to 0x2198418, (0x7cf6 bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:744 trunc from 0x9882ed9 to 0x77e2900 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:745 writesame 0x2ddd0e thru 0x2e60ed data_size 0x50(0x83e0 bytes) 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:748 trunc from 0x77e2900 to 0xb407c86 2024-04-23T23:50:03.177 INFO:teuthology.orchestra.run.smithi092.stdout:749 read 0x75d75ea thru 0x75dab02 (0x3519 bytes) 2024-04-23T23:50:03.178 INFO:teuthology.orchestra.run.smithi092.stdout:751 read 0x6da4067 thru 0x6da8679 (0x4613 bytes) 2024-04-23T23:50:03.178 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.178 INFO:teuthology.orchestra.run.smithi092.stdout:753 writesame 0x89a5307 thru 0x89a722b data_size 0x1(0x1f25 bytes) 2024-04-23T23:50:03.186 INFO:teuthology.orchestra.run.smithi092.stdout:755 trunc from 0xb407c86 to 0x1dc3b2b 2024-04-23T23:50:03.204 INFO:teuthology.orchestra.run.smithi092.stdout:756 punch from 0x103d59 to 0x110113, (0xc3ba bytes) 2024-04-23T23:50:03.204 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.204 INFO:teuthology.orchestra.run.smithi092.stdout:758 read 0x184768f thru 0x184f27f (0x7bf1 bytes) 2024-04-23T23:50:03.204 INFO:teuthology.orchestra.run.smithi092.stdout:759 read 0x5e4141 thru 0x5eae32 (0x6cf2 bytes) 2024-04-23T23:50:03.204 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.204 INFO:teuthology.orchestra.run.smithi092.stdout:762 read 0x11a3469 thru 0x11b14cc (0xe064 bytes) 2024-04-23T23:50:03.205 INFO:teuthology.orchestra.run.smithi092.stdout:763 read 0x1d17d1d thru 0x1d1e113 (0x63f7 bytes) 2024-04-23T23:50:03.205 INFO:teuthology.orchestra.run.smithi092.stdout:764 write 0x695da1 thru 0x6a2940 (0xcba0 bytes) 2024-04-23T23:50:03.217 INFO:teuthology.orchestra.run.smithi092.stdout:766 write 0x6358e5c thru 0x6368258 (0xf3fd bytes) 2024-04-23T23:50:03.223 INFO:teuthology.orchestra.run.smithi092.stdout:767 punch from 0x3c9436f to 0x3ca1b2a, (0xd7bb bytes) 2024-04-23T23:50:03.223 INFO:teuthology.orchestra.run.smithi092.stdout:768 write 0x5275eee thru 0x527d81c (0x792f bytes) 2024-04-23T23:50:03.224 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.224 INFO:teuthology.orchestra.run.smithi092.stdout:772 write 0x37c4181 thru 0x37cd3f4 (0x9274 bytes) 2024-04-23T23:50:03.228 INFO:teuthology.orchestra.run.smithi092.stdout:773 writesame 0xdd2011 thru 0xddd319 data_size 0x1(0xb309 bytes) 2024-04-23T23:50:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:774 punch from 0x48c2c39 to 0x48cc7b1, (0x9b78 bytes) 2024-04-23T23:50:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:778 read 0x679845 thru 0x685b8b (0xc347 bytes) 2024-04-23T23:50:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:779 read 0x21015c0 thru 0x2104c37 (0x3678 bytes) 2024-04-23T23:50:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:785 read 0x39b9f09 thru 0x39c3431 (0x9529 bytes) 2024-04-23T23:50:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:786 punch from 0x3f64242 to 0x3f6be85, (0x7c43 bytes) 2024-04-23T23:50:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:787 trunc from 0x6368259 to 0xb1e67ca 2024-04-23T23:50:03.270 INFO:teuthology.orchestra.run.smithi092.stdout:788 punch from 0x49f63f4 to 0x4a054ee, (0xf0fa bytes) 2024-04-23T23:50:03.271 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.271 INFO:teuthology.orchestra.run.smithi092.stdout:790 punch from 0x95c76fa to 0x95cf4d2, (0x7dd8 bytes) 2024-04-23T23:50:03.271 INFO:teuthology.orchestra.run.smithi092.stdout:791 punch from 0x74ef17a to 0x74f5446, (0x62cc bytes) 2024-04-23T23:50:03.273 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:03.287+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:03.280 INFO:teuthology.orchestra.run.smithi092.stdout:792 write 0xe8a61eb thru 0xe8b05b8 (0xa3ce bytes) 2024-04-23T23:50:03.286 INFO:teuthology.orchestra.run.smithi092.stdout:793 read 0x77a071d thru 0x77a52d5 (0x4bb9 bytes) 2024-04-23T23:50:03.286 INFO:teuthology.orchestra.run.smithi092.stdout:794 writesame 0x5ac967 thru 0x5b88be data_size 0x34(0xbf58 bytes) 2024-04-23T23:50:03.288 INFO:teuthology.orchestra.run.smithi092.stdout:795 punch from 0x721eba to 0x730fcc, (0xf112 bytes) 2024-04-23T23:50:03.288 INFO:teuthology.orchestra.run.smithi092.stdout:796 read 0xcbe70df thru 0xcbe7355 (0x277 bytes) 2024-04-23T23:50:03.288 INFO:teuthology.orchestra.run.smithi092.stdout:797 punch from 0xab59f71 to 0xab6232a, (0x83b9 bytes) 2024-04-23T23:50:03.288 INFO:teuthology.orchestra.run.smithi092.stdout:798 writesame 0x1a3cc2d thru 0x1a4a03e data_size 0xa(0xd412 bytes) 2024-04-23T23:50:03.294 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.294 INFO:teuthology.orchestra.run.smithi092.stdout:800 writesame 0xa3a0567 thru 0xa3a7ae6 data_size 0x10(0x7580 bytes) 2024-04-23T23:50:03.298 INFO:teuthology.orchestra.run.smithi092.stdout:801 read 0x8b261c0 thru 0x8b32fb2 (0xcdf3 bytes) 2024-04-23T23:50:03.301 INFO:teuthology.orchestra.run.smithi092.stdout:802 trunc from 0xe8b05b9 to 0xbdf9c46 2024-04-23T23:50:03.311 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:03.311 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:03.314 INFO:teuthology.orchestra.run.smithi092.stdout:804 read 0x738ad62 thru 0x738f332 (0x45d1 bytes) 2024-04-23T23:50:03.314 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.314 INFO:teuthology.orchestra.run.smithi092.stdout:806 writesame 0xb96fd5a thru 0xb97867b data_size 0x3(0x8922 bytes) 2024-04-23T23:50:03.320 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:03.321 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:03.323 INFO:teuthology.orchestra.run.smithi092.stdout:807 write 0x88cc76a thru 0x88d0a35 (0x42cc bytes) 2024-04-23T23:50:03.328 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.328 INFO:teuthology.orchestra.run.smithi092.stdout:809 write 0x5be3f2d thru 0x5bee1f7 (0xa2cb bytes) 2024-04-23T23:50:03.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:03.330 INFO:teuthology.orchestra.run.smithi092.stdout:810 write 0xa4b1329 thru 0xa4b5eb7 (0x4b8f bytes) 2024-04-23T23:50:03.332 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:812 read 0x9fbdc3c thru 0x9fc258d (0x4952 bytes) 2024-04-23T23:50:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:815 punch from 0x7ec888c to 0x7ed6299, (0xda0d bytes) 2024-04-23T23:50:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:817 read 0xf8ff8a thru 0xf9e625 (0xe69c bytes) 2024-04-23T23:50:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:03.333 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:03.334 INFO:teuthology.orchestra.run.smithi092.stdout:818 punch from 0xb1283c to 0xb18b77, (0x633b bytes) 2024-04-23T23:50:03.334 INFO:teuthology.orchestra.run.smithi092.stdout:819 read 0x427843f thru 0x4287075 (0xec37 bytes) 2024-04-23T23:50:03.334 INFO:teuthology.orchestra.run.smithi092.stdout:822 write 0x55b5a49 thru 0x55bef2a (0x94e2 bytes) 2024-04-23T23:50:03.346 INFO:teuthology.orchestra.run.smithi092.stdout:824 trunc from 0xbdf9c46 to 0x39f8226 2024-04-23T23:50:03.352 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:03.372 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:03.372 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:03.372 INFO:teuthology.orchestra.run.smithi092.stdout:825 write 0x8178a9b thru 0x81799cf (0xf35 bytes) 2024-04-23T23:50:03.374 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:03.387+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:03.385 INFO:teuthology.orchestra.run.smithi092.stdout:826 write 0xc4257a9 thru 0xc432230 (0xca88 bytes) 2024-04-23T23:50:03.388 INFO:teuthology.orchestra.run.smithi092.stdout:827 read 0xaadc244 thru 0xaae5c40 (0x99fd bytes) 2024-04-23T23:50:03.388 INFO:teuthology.orchestra.run.smithi092.stdout:828 write 0x1a91ddf thru 0x1a9e032 (0xc254 bytes) 2024-04-23T23:50:03.390 INFO:teuthology.orchestra.run.smithi092.stdout:829 read 0x7be04bc thru 0x7beb6bf (0xb204 bytes) 2024-04-23T23:50:03.390 INFO:teuthology.orchestra.run.smithi092.stdout:830 writesame 0x1812673 thru 0x1816f5f data_size 0x31(0x48ed bytes) 2024-04-23T23:50:03.391 INFO:teuthology.orchestra.run.smithi092.stdout:831 write 0x6cffecd thru 0x6d0c0f1 (0xc225 bytes) 2024-04-23T23:50:03.392 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:03.392 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:03.395 INFO:teuthology.orchestra.run.smithi092.stdout:832 punch from 0xbe9c86a to 0xbea136e, (0x4b04 bytes) 2024-04-23T23:50:03.407 INFO:teuthology.orchestra.run.smithi092.stdout:833 trunc from 0xc432231 to 0x2311807 2024-04-23T23:50:03.412 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:03.412 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:03.422 INFO:teuthology.orchestra.run.smithi092.stdout:834 read 0x2075929 thru 0x2079ad0 (0x41a8 bytes) 2024-04-23T23:50:03.428 INFO:teuthology.orchestra.run.smithi092.stdout:835 write 0x3317c4c thru 0x33276d4 (0xfa89 bytes) 2024-04-23T23:50:03.431 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:03.432 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.432 INFO:teuthology.orchestra.run.smithi092.stdout:837 write 0x31b185e thru 0x31c0914 (0xf0b7 bytes) 2024-04-23T23:50:03.435 INFO:teuthology.orchestra.run.smithi092.stdout:838 read 0x15fe1f2 thru 0x1603517 (0x5326 bytes) 2024-04-23T23:50:03.435 INFO:teuthology.orchestra.run.smithi092.stdout:839 punch from 0x1c4a0b6 to 0x1c58e31, (0xed7b bytes) 2024-04-23T23:50:03.450 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:03.450 INFO:teuthology.orchestra.run.smithi092.stdout:841 write 0xa36f101 thru 0xa373178 (0x4078 bytes) 2024-04-23T23:50:03.454 INFO:teuthology.orchestra.run.smithi092.stdout:842 write 0x6a519f6 thru 0x6a5ee34 (0xd43f bytes) 2024-04-23T23:50:03.457 INFO:teuthology.orchestra.run.smithi092.stdout:843 read 0x95b0f5b thru 0x95bb909 (0xa9af bytes) 2024-04-23T23:50:03.457 INFO:teuthology.orchestra.run.smithi092.stdout:844 punch from 0x960526b to 0x9615194, (0xff29 bytes) 2024-04-23T23:50:03.458 INFO:teuthology.orchestra.run.smithi092.stdout:845 trunc from 0xa373179 to 0x93be16e 2024-04-23T23:50:03.468 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.469 INFO:teuthology.orchestra.run.smithi092.stdout:848 write 0x420bb80 thru 0x421a4da (0xe95b bytes) 2024-04-23T23:50:03.470 INFO:teuthology.orchestra.run.smithi092.stdout:849 write 0x48c59e5 thru 0x48ca286 (0x48a2 bytes) 2024-04-23T23:50:03.473 INFO:teuthology.orchestra.run.smithi092.stdout:850 write 0x16c7c9b thru 0x16cc1c2 (0x4528 bytes) 2024-04-23T23:50:03.474 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:03.488+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:03.474 INFO:teuthology.orchestra.run.smithi092.stdout:851 read 0x5282a9c thru 0x52905f8 (0xdb5d bytes) 2024-04-23T23:50:03.482 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:03.482 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:03.483 INFO:teuthology.orchestra.run.smithi092.stdout:853 writesame 0xc1590ab thru 0xc15b719 data_size 0x1(0x266f bytes) 2024-04-23T23:50:03.488 INFO:teuthology.orchestra.run.smithi092.stdout:854 trunc from 0xc15b71a to 0x87d2b49 2024-04-23T23:50:03.492 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:03.493 INFO:teuthology.orchestra.run.smithi092.stdout:855 write 0x82adde5 thru 0x82b8d9e (0xafba bytes) 2024-04-23T23:50:03.498 INFO:teuthology.orchestra.run.smithi092.stdout:856 punch from 0x64e989 to 0x65df96, (0xf60d bytes) 2024-04-23T23:50:03.498 INFO:teuthology.orchestra.run.smithi092.stdout:857 read 0x19e4fd2 thru 0x19f4530 (0xf55f bytes) 2024-04-23T23:50:03.506 INFO:teuthology.orchestra.run.smithi092.stdout:858 write 0xa2bd0d0 thru 0xa2cc281 (0xf1b2 bytes) 2024-04-23T23:50:03.510 INFO:teuthology.orchestra.run.smithi092.stdout:859 write 0x3eb3c09 thru 0x3eb3da6 (0x19e bytes) 2024-04-23T23:50:03.516 INFO:teuthology.orchestra.run.smithi092.stdout:860 trunc from 0xa2cc282 to 0x74cca96 2024-04-23T23:50:03.528 INFO:teuthology.orchestra.run.smithi092.stdout:861 read 0x2e1625f thru 0x2e232dd (0xd07f bytes) 2024-04-23T23:50:03.528 INFO:teuthology.orchestra.run.smithi092.stdout:862 read 0x5cb9c18 thru 0x5cc1581 (0x796a bytes) 2024-04-23T23:50:03.528 INFO:teuthology.orchestra.run.smithi092.stdout:865 write 0x3174d42 thru 0x3179c45 (0x4f04 bytes) 2024-04-23T23:50:03.532 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.532 INFO:teuthology.orchestra.run.smithi092.stdout:868 punch from 0x600850b to 0x600f7ee, (0x72e3 bytes) 2024-04-23T23:50:03.543 INFO:teuthology.orchestra.run.smithi092.stdout:870 write 0xb7d5993 thru 0xb7df825 (0x9e93 bytes) 2024-04-23T23:50:03.550 INFO:teuthology.orchestra.run.smithi092.stdout:871 trunc from 0xb7df826 to 0x7cae5c2 2024-04-23T23:50:03.560 INFO:teuthology.orchestra.run.smithi092.stdout:872 read 0x53aadb2 thru 0x53ba620 (0xf86f bytes) 2024-04-23T23:50:03.560 INFO:teuthology.orchestra.run.smithi092.stdout:875 write 0x6c3e54c thru 0x6c4c07a (0xdb2f bytes) 2024-04-23T23:50:03.563 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.575 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:03.589+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:03.575 INFO:teuthology.orchestra.run.smithi092.stdout:877 writesame 0xd956077 thru 0xd959c25 data_size 0xb(0x3baf bytes) 2024-04-23T23:50:03.576 INFO:teuthology.orchestra.run.smithi092.stdout:878 write 0x1c8e05b thru 0x1c91478 (0x341e bytes) 2024-04-23T23:50:03.587 INFO:teuthology.orchestra.run.smithi092.stdout:879 trunc from 0xd959c26 to 0x4f72c0d 2024-04-23T23:50:03.603 INFO:teuthology.orchestra.run.smithi092.stdout:880 read 0x416c134 thru 0x4179cc0 (0xdb8d bytes) 2024-04-23T23:50:03.603 INFO:teuthology.orchestra.run.smithi092.stdout:881 punch from 0x5f0e14 to 0x5fa84e, (0x9a3a bytes) 2024-04-23T23:50:03.616 INFO:teuthology.orchestra.run.smithi092.stdout:882 writesame 0x8728c33 thru 0x873177b data_size 0x1(0x8b49 bytes) 2024-04-23T23:50:03.616 INFO:teuthology.orchestra.run.smithi092.stdout:883 punch from 0x5e15da9 to 0x5e16037, (0x28e bytes) 2024-04-23T23:50:03.620 INFO:teuthology.orchestra.run.smithi092.stdout:884 writesame 0x87d9014 thru 0x87e7f33 data_size 0x10(0xef20 bytes) 2024-04-23T23:50:03.620 INFO:teuthology.orchestra.run.smithi092.stdout:885 read 0xe93c05 thru 0xe9b7e9 (0x7be5 bytes) 2024-04-23T23:50:03.621 INFO:teuthology.orchestra.run.smithi092.stdout:886 read 0x54cb25a thru 0x54d2eec (0x7c93 bytes) 2024-04-23T23:50:03.621 INFO:teuthology.orchestra.run.smithi092.stdout:887 punch from 0x18692df to 0x18761a6, (0xcec7 bytes) 2024-04-23T23:50:03.621 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.621 INFO:teuthology.orchestra.run.smithi092.stdout:889 write 0x802c5d0 thru 0x803aaac (0xe4dd bytes) 2024-04-23T23:50:03.627 INFO:teuthology.orchestra.run.smithi092.stdout:891 write 0x26de10b thru 0x26e4fdb (0x6ed1 bytes) 2024-04-23T23:50:03.637 INFO:teuthology.orchestra.run.smithi092.stdout:892 trunc from 0x87e7f34 to 0xd4d9fc 2024-04-23T23:50:03.640 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:03.640 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:03.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:03.671 INFO:teuthology.orchestra.run.smithi092.stdout:895 writesame 0x90a2f47 thru 0x90a4922 data_size 0xa(0x19dc bytes) 2024-04-23T23:50:03.671 INFO:teuthology.orchestra.run.smithi092.stdout:896 flatten 2024-04-23T23:50:03.676 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:03.690+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:03.677 INFO:teuthology.orchestra.run.smithi092.stdout:897 write 0x1331ec1 thru 0x1340405 (0xe545 bytes) 2024-04-23T23:50:03.685 INFO:teuthology.orchestra.run.smithi092.stdout:898 write 0xa7238fb thru 0xa72c8a5 (0x8fab bytes) 2024-04-23T23:50:03.688 INFO:teuthology.orchestra.run.smithi092.stdout:899 read 0x7d5918b thru 0x7d5c338 (0x31ae bytes) 2024-04-23T23:50:03.688 INFO:teuthology.orchestra.run.smithi092.stdout:901 read 0xaba288 thru 0xac8979 (0xe6f2 bytes) 2024-04-23T23:50:03.688 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.693 INFO:teuthology.orchestra.run.smithi092.stdout:903 write 0xab7448a thru 0xab7ee11 (0xa988 bytes) 2024-04-23T23:50:03.696 INFO:teuthology.orchestra.run.smithi092.stdout:905 punch from 0x2fb0dbc to 0x2fb347f, (0x26c3 bytes) 2024-04-23T23:50:03.696 INFO:teuthology.orchestra.run.smithi092.stdout:906 writesame 0x8935a7e thru 0x89459b2 data_size 0x1(0xff35 bytes) 2024-04-23T23:50:03.731 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.732 INFO:teuthology.orchestra.run.smithi092.stdout:908 read 0x6a1776e thru 0x6a25599 (0xde2c bytes) 2024-04-23T23:50:03.745 INFO:teuthology.orchestra.run.smithi092.stdout:909 trunc from 0xab7ee12 to 0x3210cd 2024-04-23T23:50:03.767 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:03.767 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:03.777 INFO:teuthology.orchestra.run.smithi092.stdout:910 write 0x866d8bb thru 0x8671132 (0x3878 bytes) 2024-04-23T23:50:03.777 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:03.791+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:03.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:911 write 0x13eca66 thru 0x13eff91 (0x352c bytes) 2024-04-23T23:50:03.792 INFO:teuthology.orchestra.run.smithi092.stdout:912 trunc from 0x8671133 to 0x84c992 2024-04-23T23:50:03.820 INFO:teuthology.orchestra.run.smithi092.stdout:913 trunc from 0x84c992 to 0xbb8f8e5 2024-04-23T23:50:03.826 INFO:teuthology.orchestra.run.smithi092.stdout:915 write 0xb20db12 thru 0xb2159ee (0x7edd bytes) 2024-04-23T23:50:03.829 INFO:teuthology.orchestra.run.smithi092.stdout:917 write 0x393ab97 thru 0x393b04f (0x4b9 bytes) 2024-04-23T23:50:03.834 INFO:teuthology.orchestra.run.smithi092.stdout:918 trunc from 0xbb8f8e5 to 0xd42f609 2024-04-23T23:50:03.839 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:03.839 INFO:teuthology.orchestra.run.smithi092.stdout:920 read 0xad07dd4 thru 0xad120c2 (0xa2ef bytes) 2024-04-23T23:50:03.854 INFO:teuthology.orchestra.run.smithi092.stdout:921 trunc from 0xd42f609 to 0x17928dc 2024-04-23T23:50:03.878 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:03.892+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:03.880 INFO:teuthology.orchestra.run.smithi092.stdout:922 write 0x3aa28c3 thru 0x3aa981b (0x6f59 bytes) 2024-04-23T23:50:03.886 INFO:teuthology.orchestra.run.smithi092.stdout:923 read 0x35ad8be thru 0x35b9dd8 (0xc51b bytes) 2024-04-23T23:50:03.887 INFO:teuthology.orchestra.run.smithi092.stdout:924 trunc from 0x3aa981c to 0x658596 2024-04-23T23:50:03.900 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:03.900 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:03.901 INFO:teuthology.orchestra.run.smithi092.stdout:925 punch from 0x2d96ff to 0x2e60eb, (0xc9ec bytes) 2024-04-23T23:50:03.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:03.913 INFO:teuthology.orchestra.run.smithi092.stdout:928 write 0x4d2a854 thru 0x4d2eb97 (0x4344 bytes) 2024-04-23T23:50:03.928 INFO:teuthology.orchestra.run.smithi092.stdout:929 write 0xbed710f thru 0xbee1fec (0xaede bytes) 2024-04-23T23:50:03.932 INFO:teuthology.orchestra.run.smithi092.stdout:930 punch from 0x9ce6934 to 0x9cf47a6, (0xde72 bytes) 2024-04-23T23:50:03.941 INFO:teuthology.orchestra.run.smithi092.stdout:931 writesame 0xec08866 thru 0xec143f1 data_size 0x6(0xbb8c bytes) 2024-04-23T23:50:03.941 INFO:teuthology.orchestra.run.smithi092.stdout:932 read 0x8972507 thru 0x897a382 (0x7e7c bytes) 2024-04-23T23:50:03.941 INFO:teuthology.orchestra.run.smithi092.stdout:933 write 0x9bf2366 thru 0x9bf25e6 (0x281 bytes) 2024-04-23T23:50:03.944 INFO:teuthology.orchestra.run.smithi092.stdout:934 write 0x48d79f3 thru 0x48de5bf (0x6bcd bytes) 2024-04-23T23:50:03.947 INFO:teuthology.orchestra.run.smithi092.stdout:935 writesame 0x139790b thru 0x13a56f6 data_size 0xe(0xddec bytes) 2024-04-23T23:50:03.951 INFO:teuthology.orchestra.run.smithi092.stdout:936 read 0xb197f54 thru 0xb19dec9 (0x5f76 bytes) 2024-04-23T23:50:03.951 INFO:teuthology.orchestra.run.smithi092.stdout:937 read 0x55e5ad3 thru 0x55f3409 (0xd937 bytes) 2024-04-23T23:50:03.951 INFO:teuthology.orchestra.run.smithi092.stdout:938 read 0x66e6362 thru 0x66e8b1b (0x27ba bytes) 2024-04-23T23:50:03.958 INFO:teuthology.orchestra.run.smithi092.stdout:939 trunc from 0xec143f2 to 0x86525ab 2024-04-23T23:50:03.979 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:03.993+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:03.980 INFO:teuthology.orchestra.run.smithi092.stdout:940 write 0xcea6f44 thru 0xceac676 (0x5733 bytes) 2024-04-23T23:50:03.984 INFO:teuthology.orchestra.run.smithi092.stdout:942 punch from 0x67a9d34 to 0x67b906a, (0xf336 bytes) 2024-04-23T23:50:03.984 INFO:teuthology.orchestra.run.smithi092.stdout:943 writesame 0x18013d9 thru 0x180fa72 data_size 0x3(0xe69a bytes) 2024-04-23T23:50:03.996 INFO:teuthology.orchestra.run.smithi092.stdout:944 writesame 0x4123291 thru 0x4129206 data_size 0x3(0x5f76 bytes) 2024-04-23T23:50:04.016 INFO:teuthology.orchestra.run.smithi092.stdout:947 trunc from 0xceac677 to 0x33c17e5 2024-04-23T23:50:04.030 INFO:teuthology.orchestra.run.smithi092.stdout:948 read 0x29380fa thru 0x293d245 (0x514c bytes) 2024-04-23T23:50:04.031 INFO:teuthology.orchestra.run.smithi092.stdout:949 punch from 0x1513212 to 0x15143cf, (0x11bd bytes) 2024-04-23T23:50:04.031 INFO:teuthology.orchestra.run.smithi092.stdout:950 punch from 0x33732d to 0x33d294, (0x5f67 bytes) 2024-04-23T23:50:04.036 INFO:teuthology.orchestra.run.smithi092.stdout:951 trunc from 0x33c17e5 to 0x7cf9030 2024-04-23T23:50:04.041 INFO:teuthology.orchestra.run.smithi092.stdout:952 read 0x11ce3aa thru 0x11d0508 (0x215f bytes) 2024-04-23T23:50:04.041 INFO:teuthology.orchestra.run.smithi092.stdout:954 read 0x4581b5 thru 0x460227 (0x8073 bytes) 2024-04-23T23:50:04.042 INFO:teuthology.orchestra.run.smithi092.stdout:955 clone 8 order 21 su 1048576 sc 9 2024-04-23T23:50:04.049 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.050 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:04.081 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:04.094+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:04.182 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:04.196+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:04.203 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.203 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:04.283 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:04.297+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:04.354 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.355 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:04.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:50:04.384 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:04.398+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:04.460 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-23T23:50:04.460 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:04.460 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-04-23T23:50:04.460 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-23T23:50:04.460 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-23T23:50:04.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-04-23T23:50:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:04.504 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:04.518+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:04.565 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:04.565 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-23T23:50:04.565 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:04.586 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:04.600+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:04.598 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.598 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:04.687 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:04.701+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:04.751 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.751 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:04.804 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:04.818+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:04.861 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:04.862 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:04.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:04.911 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:04.925+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:50:04.990 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:05.004+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:05.024 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:05.024 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:05.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:05.091 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:05.104+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:05.093 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-04-23T23:50:05.105 INFO:teuthology.orchestra.run.smithi092.stdout:956 punch from 0x6783e34 to 0x678852d, (0x46f9 bytes) 2024-04-23T23:50:05.109 INFO:teuthology.orchestra.run.smithi092.stdout:957 clone 9 order 25 su 131072 sc 5 2024-04-23T23:50:05.142 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:05.142 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:05.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:05.192 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:05.205+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:05.292 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:05.306+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:05.293 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:05.293 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:05.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:05.393 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:05.407+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:05.405 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:05.406 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:05.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:05.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:05.508 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:05.522+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:05.515 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:05.516 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:05.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:05.583 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:05.594 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:05.608+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:05.600 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:05.617 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:05.617 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:05.626 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:05.627 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:05.634 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:05.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:05.650 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:05.651 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:05.667 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:05.668 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:05.686 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:05.686 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:05.695 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:05.709+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:05.703 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:05.739 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:05.739 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:05.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:05.797 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:05.810+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:05.848 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:05.848 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:05.848 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:05.848 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:05.848 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:05.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:05.900 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:05.914+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:542 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0833078240000003, 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065200570000000001, 2024-04-23T23:50:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 542 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893962+0000", 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.58e-07 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893965+0000", 2024-04-23T23:50:05.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9280000000000002e-06 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893973+0000", 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2729999999999997e-06 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893987+0000", 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4344e-05 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895263+0000", 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012758610000000001 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895316+0000", 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3724999999999998e-05 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895563+0000", 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024641700000000002 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898014+0000", 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002451103 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900408+0000", 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00239419 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900456+0000", 2024-04-23T23:50:05.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7837999999999998e-05 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900482+0000", 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5919999999999999e-05 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:543 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0832718010000004, 2024-04-23T23:50:05.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065321299999999997, 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 543 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.893998+0000", 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.34e-07 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894000+0000", 2024-04-23T23:50:05.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1100000000000001e-06 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894005+0000", 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.126e-06 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894009+0000", 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895875+0000", 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018654260000000001 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.895926+0000", 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1221000000000003e-05 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896212+0000", 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028643100000000002 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898050+0000", 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001837279 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900488+0000", 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024382330000000002 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900512+0000", 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3818000000000001e-05 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900530+0000", 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8278999999999999e-05 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:544 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:05.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0832556479999997, 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006558014, 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 544 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:05.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894014+0000", 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894016+0000", 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.393e-06 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894019+0000", 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2559999999999998e-06 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894022+0000", 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8490000000000002e-06 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896641+0000", 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026193380000000001 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896701+0000", 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0019000000000003e-05 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.896940+0000", 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023890900000000001 2024-04-23T23:50:05.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898076+0000", 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001136242 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900534+0000", 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024572869999999998 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900555+0000", 2024-04-23T23:50:05.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1872999999999999e-05 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.900572+0000", 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6578999999999998e-05 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:545 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.083244702, 2024-04-23T23:50:05.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091656180000000004, 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 545 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894025+0000", 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894027+0000", 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.446e-06 2024-04-23T23:50:05.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894030+0000", 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.134e-06 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.894033+0000", 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3950000000000001e-06 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.897312+0000", 2024-04-23T23:50:05.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003278572 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.897367+0000", 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5566000000000003e-05 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.897694+0000", 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00032677100000000001 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.898554+0000", 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000859953 2024-04-23T23:50:05.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903118+0000", 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045641759999999996 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903158+0000", 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9530000000000003e-05 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.903191+0000", 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2854000000000001e-05 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:572 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921286+0000", 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0559833110000003, 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073967379999999999, 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:05.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 572 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921286+0000", 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:05.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921286+0000", 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921287+0000", 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7e-07 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921289+0000", 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.086e-06 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921293+0000", 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.386e-06 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921297+0000", 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3349999999999998e-06 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922700+0000", 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001403577 2024-04-23T23:50:05.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.922784+0000", 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418999999999995e-05 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923159+0000", 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037584899999999999 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:05.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924999+0000", 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001839799 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928595+0000", 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035961399999999998 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928644+0000", 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8804000000000002e-05 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928683+0000", 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8973000000000001e-05 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:573 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.055951812, 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074509830000000004, 2024-04-23T23:50:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 573 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921318+0000", 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:05.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921320+0000", 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0930000000000001e-06 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921325+0000", 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1389999999999997e-06 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921328+0000", 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0489999999999999e-06 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923614+0000", 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022866890000000002 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923665+0000", 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0661000000000002e-05 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.923878+0000", 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021355199999999999 2024-04-23T23:50:05.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.926772+0000", 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028935850000000002 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928693+0000", 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001920997 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928734+0000", 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1304999999999998e-05 2024-04-23T23:50:06.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928769+0000", 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4505999999999998e-05 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:575 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921379+0000", 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0558903740000005, 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074341980000000004, 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 575 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921379+0000", 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921379+0000", 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921380+0000", 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4200000000000002e-07 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921381+0000", 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.832e-06 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921385+0000", 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921388+0000", 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9110000000000002e-06 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924196+0000", 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028076749999999999 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924274+0000", 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7886999999999995e-05 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.924629+0000", 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035520300000000001 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.927147+0000", 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025179619999999999 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928774+0000", 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001626976 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928795+0000", 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1112e-05 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928814+0000", 2024-04-23T23:50:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8731000000000001e-05 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:576 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0558645640000002, 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074581659999999996, 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 576 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921405+0000", 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921407+0000", 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8589999999999999e-06 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921411+0000", 2024-04-23T23:50:06.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9380000000000002e-06 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.921414+0000", 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0290000000000001e-06 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.925028+0000", 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003613808 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.925109+0000", 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0989000000000007e-05 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.925461+0000", 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035196400000000002 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.927186+0000", 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017245610000000001 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928817+0000", 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001631696 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928844+0000", 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6976000000000001e-05 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.928863+0000", 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8995999999999999e-05 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:638 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.991276+0000", 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.985993616, 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0098125250000000008, 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 638 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.013 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991276+0000", 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991276+0000", 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991277+0000", 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3399999999999999e-07 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991280+0000", 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.281e-06 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991289+0000", 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7790000000000001e-06 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991294+0000", 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5740000000000003e-06 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992754+0000", 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001459498 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992844+0000", 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0205999999999997e-05 2024-04-23T23:50:06.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.993212+0000", 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036836300000000002 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.995097+0000", 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018850049999999999 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001033+0000", 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059358240000000001 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001065+0000", 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1511999999999999e-05 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001089+0000", 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3849000000000002e-05 2024-04-23T23:50:06.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:639 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.991360+0000", 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9859092179999998, 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097745739999999994, 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 639 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991360+0000", 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991360+0000", 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991361+0000", 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999991e-07 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991374+0000", 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2428e-05 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991385+0000", 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0694e-05 2024-04-23T23:50:06.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.991392+0000", 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.002e-06 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.993743+0000", 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023511000000000001 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.993792+0000", 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9607999999999997e-05 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994000+0000", 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000207846 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.995962+0000", 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019620050000000002 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001095+0000", 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051327389999999999 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001118+0000", 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3091e-05 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001135+0000", 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7101000000000001e-05 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.024 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:640 3.c 3:35136f61:::rbd_object_map.1179d7437ec6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:56.992430+0000", 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9848395310000004, 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087474370000000003, 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 640 2024-04-23T23:50:06.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992430+0000", 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992430+0000", 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992431+0000", 2024-04-23T23:50:06.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.012e-06 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992437+0000", 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3759999999999999e-06 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992449+0000", 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2408999999999999e-05 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.992457+0000", 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1799999999999996e-06 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994312+0000", 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018544049999999999 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994391+0000", 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9258999999999994e-05 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.994748+0000", 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035705200000000001 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:56.996175+0000", 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00142716 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001139+0000", 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049635410000000001 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001160+0000", 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1676e-05 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.001178+0000", 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7367000000000002e-05 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.030 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1145 3.17 e29/22 3:ef88a9a9:::rbd_data.1179984961fa.0000000000000001:head v 29'9, mlcod=29'7)", 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.924079+0000", 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0531906039999992, 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013902618, 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924079+0000", 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924079+0000", 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924081+0000", 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.513e-06 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928877+0000", 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047967950000000004 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928895+0000", 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7642000000000002e-05 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928904+0000", 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1870000000000003e-06 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928976+0000", 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1870000000000001e-05 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.928995+0000", 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8915000000000001e-05 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937940+0000", 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089449739999999996 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937982+0000", 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1721999999999998e-05 2024-04-23T23:50:06.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1149 3.17 e29/22 3:e91d246d:::rbd_data.1179984961fa.0000000000000008:head v 29'10, mlcod=29'7)", 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.930374+0000", 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0468957569999997, 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076889610000000002, 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930374+0000", 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930374+0000", 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930375+0000", 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9500000000000001e-07 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4279999999999999e-06 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930396+0000", 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3487e-05 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930404+0000", 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.931651+0000", 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001247492 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.931671+0000", 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0290999999999999e-05 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938028+0000", 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006357003 2024-04-23T23:50:06.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938063+0000", 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.447e-05 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3436 3.1d 3:b8481f77:::rbd_data.1179c62058ea.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.935334+0000", 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.0419356449999997, 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005238581, 2024-04-23T23:50:06.039 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3436 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935334+0000", 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935334+0000", 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935335+0000", 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4300000000000001e-06 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935340+0000", 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0089999999999998e-06 2024-04-23T23:50:06.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935355+0000", 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4728e-05 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935363+0000", 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8790000000000003e-06 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935446+0000", 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3150000000000002e-05 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935521+0000", 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4578000000000003e-05 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.935975+0000", 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000454673 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940473+0000", 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044979110000000003 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940511+0000", 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7521000000000002e-05 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940541+0000", 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0219999999999999e-05 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.940573+0000", 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1482000000000001e-05 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3596 3.1d 3:bafdd96b:::rbd_data.1179c62058ea.0000000000000002:head [stat,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.045 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.019892+0000", 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9573780689999998, 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051972729999999997, 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3596 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019892+0000", 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019892+0000", 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019893+0000", 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6169999999999999e-06 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019901+0000", 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4730000000000003e-06 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019925+0000", 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4632000000000001e-05 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019935+0000", 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7839999999999998e-06 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020013+0000", 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8333999999999999e-05 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020108+0000", 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4852000000000006e-05 2024-04-23T23:50:06.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020251+0000", 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000142922 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.021103+0000", 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085139899999999997 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025002+0000", 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038996510000000001 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025051+0000", 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8349000000000001e-05 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025089+0000", 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8260000000000003e-05 2024-04-23T23:50:06.051 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3604 3.1c 3:3a1edd37:::rbd_data.1179c62058ea.000000000000000a:head [stat,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.020035+0000", 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.957234696, 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059299970000000002, 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.052 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3604 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020035+0000", 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020035+0000", 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020036+0000", 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3799999999999997e-07 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020039+0000", 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8070000000000001e-06 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020048+0000", 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8340000000000005e-06 2024-04-23T23:50:06.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020055+0000", 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3990000000000001e-06 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020114+0000", 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9358000000000003e-05 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020195+0000", 2024-04-23T23:50:06.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0679000000000005e-05 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020323+0000", 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012818899999999999 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.021120+0000", 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00079704000000000005 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025908+0000", 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047881369999999996 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025939+0000", 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0539e-05 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025965+0000", 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6477000000000001e-05 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3870 3.1d 3:bafdd96b:::rbd_data.1179c62058ea.0000000000000002:head [stat,truncate 524288] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.464629+0000", 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.5126402670000001, 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068574070000000003, 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3870 2024-04-23T23:50:06.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464629+0000", 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464629+0000", 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464631+0000", 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8109999999999999e-06 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464638+0000", 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5300000000000002e-06 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464652+0000", 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4199000000000001e-05 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464669+0000", 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7070000000000001e-05 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464733+0000", 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3765999999999999e-05 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464831+0000", 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8249000000000001e-05 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.465019+0000", 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018749399999999999 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.466959+0000", 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019400839999999999 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471382+0000", 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044238610000000003 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471439+0000", 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6907000000000003e-05 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471487+0000", 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7435999999999998e-05 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5345 3.5 3:a058b887:::rbd_data.1179c4f11774.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-23T23:50:06.063 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.945852+0000", 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.0314178389999999, 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071569099999999998, 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5345 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945852+0000", 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945852+0000", 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945853+0000", 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5629999999999999e-06 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945860+0000", 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000003e-06 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945877+0000", 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6731999999999998e-05 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945887+0000", 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8539999999999992e-06 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945942+0000", 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4929000000000002e-05 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945949+0000", 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3150000000000003e-06 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948253+0000", 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023041379999999998 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948326+0000", 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2280000000000006e-05 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948420+0000", 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4823999999999999e-05 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950362+0000", 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001941765 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.952919+0000", 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002556684 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.952971+0000", 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1675000000000002e-05 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.953009+0000", 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8173999999999997e-05 2024-04-23T23:50:06.069 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5353 3.c 3:33e951f3:::rbd_data.1179c4f11774.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.946013+0000", 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.0312570669999999, 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069961390000000002, 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5353 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946013+0000", 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946013+0000", 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946013+0000", 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3900000000000004e-07 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946017+0000", 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8979999999999998e-06 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946028+0000", 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1229999999999999e-05 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946035+0000", 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3230000000000003e-06 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946081+0000", 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6712999999999998e-05 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946087+0000", 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9429999999999997e-06 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947984+0000", 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018965469999999999 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948052+0000", 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8261000000000003e-05 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.949238+0000", 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011859419999999999 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950594+0000", 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013562030000000001 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.952930+0000", 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002335705 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.952974+0000", 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4101999999999999e-05 2024-04-23T23:50:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.953009+0000", 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4632999999999999e-05 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5354 3.5 3:a7b46cc4:::rbd_data.1179c4f11774.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.946051+0000", 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.031218752, 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070946150000000003, 2024-04-23T23:50:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5354 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946051+0000", 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946051+0000", 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946051+0000", 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9200000000000001e-07 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946054+0000", 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6879999999999999e-06 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946061+0000", 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2599999999999999e-06 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946067+0000", 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3449999999999996e-06 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946116+0000", 2024-04-23T23:50:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9475999999999999e-05 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946122+0000", 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948879+0000", 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027568610000000002 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948955+0000", 2024-04-23T23:50:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6050999999999994e-05 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.949074+0000", 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119366 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950518+0000", 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014439069999999999 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.953056+0000", 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025371809999999999 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.953104+0000", 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8705999999999998e-05 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.953146+0000", 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1267000000000003e-05 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.083 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:06.083 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone8 from 0x7cf9030 (overlap 0x7cf9030) to 0x3e7c818 2024-04-23T23:50:06.084 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:06.085 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:06.098+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:06.113 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-04-23T23:50:06.185 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:06.199+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:06.227 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:06.227 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:06.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:06.286 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:06.300+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:958 trunc from 0x7cf9030 to 0x17c6665 2024-04-23T23:50:06.356 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:06.356 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:06.356 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:06.356 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:06.356 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:06.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:06.369 INFO:teuthology.orchestra.run.smithi092.stdout:959 punch from 0xd5892e to 0xd610c4, (0x8796 bytes) 2024-04-23T23:50:06.386 INFO:teuthology.orchestra.run.smithi092.stdout:961 writesame 0xa4a6637 thru 0xa4acc47 data_size 0x1d(0x6611 bytes) 2024-04-23T23:50:06.439 INFO:teuthology.orchestra.run.smithi092.stdout:962 writesame 0x99c1b94 thru 0x99c98ff data_size 0x4(0x7d6c bytes) 2024-04-23T23:50:06.441 INFO:teuthology.orchestra.run.smithi092.stdout:963 write 0x1c9b8b0 thru 0x1c9c167 (0x8b8 bytes) 2024-04-23T23:50:06.441 INFO:teuthology.orchestra.run.smithi092.stdout:964 read 0x8b29f4 thru 0x8b565b (0x2c68 bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:965 write 0x3b837c7 thru 0x3b84109 (0x943 bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:966 read 0x8719cf8 thru 0x871bed1 (0x21da bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:967 punch from 0x156f635 to 0x156fc40, (0x60b bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:968 read 0x1288bb1 thru 0x1290b0d (0x7f5d bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:970 trunc from 0xa4acc48 to 0x7c11ea6 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:971 write 0x5dc060 thru 0x5e0364 (0x4305 bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:972 write 0x8e810eb thru 0x8e85738 (0x464e bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:973 writesame 0x4d84af9 thru 0x4d8a80e data_size 0x5(0x5d16 bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:974 write 0xceade7b thru 0xceb6764 (0x88ea bytes) 2024-04-23T23:50:06.442 INFO:teuthology.orchestra.run.smithi092.stdout:976 read 0xaf5d2e7 thru 0xaf68d7a (0xba94 bytes) 2024-04-23T23:50:06.443 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:06.455+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:06.445 INFO:teuthology.orchestra.run.smithi092.stdout:977 trunc from 0xceb6765 to 0x246446c 2024-04-23T23:50:06.457 INFO:teuthology.orchestra.run.smithi092.stdout:978 punch from 0x81e380 to 0x8291b9, (0xae39 bytes) 2024-04-23T23:50:06.457 INFO:teuthology.orchestra.run.smithi092.stdout:980 read 0x334be3 thru 0x336317 (0x1735 bytes) 2024-04-23T23:50:06.463 INFO:teuthology.orchestra.run.smithi092.stdout:981 write 0x389eefb thru 0x38a1d76 (0x2e7c bytes) 2024-04-23T23:50:06.465 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xed8ee79 2024-04-23T23:50:06.474 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:06.474 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:06.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:06.474 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:06.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.474 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1148 3.a 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15728640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.929815+0000", 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5573174899999991, 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075606140000000002, 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1148 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929815+0000", 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929815+0000", 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929817+0000", 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1600000000000001e-06 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929832+0000", 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5034e-05 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929854+0000", 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1345999999999999e-05 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929863+0000", 2024-04-23T23:50:06.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3209999999999994e-06 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929926+0000", 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3570000000000003e-05 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930074+0000", 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014782599999999999 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930283+0000", 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020858899999999999 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.933376+0000", 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003093284 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937270+0000", 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038939000000000001 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937333+0000", 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3177999999999997e-05 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937376+0000", 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2406e-05 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1445 3.a 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head [truncate 6553600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:58.299738+0000", 2024-04-23T23:50:06.480 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1873944349999999, 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039146060000000002, 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1445 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299738+0000", 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299738+0000", 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299740+0000", 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.471e-06 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299746+0000", 2024-04-23T23:50:06.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9970000000000004e-06 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299760+0000", 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4100000000000001e-05 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299768+0000", 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299846+0000", 2024-04-23T23:50:06.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7787999999999995e-05 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.299934+0000", 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8480999999999995e-05 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300062+0000", 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012774100000000001 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.300874+0000", 2024-04-23T23:50:06.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081191300000000004 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303572+0000", 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002698776 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303623+0000", 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0408000000000003e-05 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:58.303653+0000", 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9782e-05 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3322 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 16252928~60550] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870711+0000", 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6164210199999998, 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042916940000000004, 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.486 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3322 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870711+0000", 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870711+0000", 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870712+0000", 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3500000000000002e-07 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870714+0000", 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.08e-06 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870731+0000", 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7337e-05 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870737+0000", 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874270+0000", 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003532898 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.874359+0000", 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8819999999999993e-05 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875003+0000", 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00064445699999999997 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.490 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3343 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.870988+0000", 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6161440779999996, 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042296779999999997, 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3343 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870988+0000", 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870988+0000", 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870989+0000", 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9100000000000003e-07 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.870993+0000", 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.092e-06 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871003+0000", 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8470000000000008e-06 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871010+0000", 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3590000000000002e-06 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875037+0000", 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040269069999999997 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875118+0000", 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0422999999999999e-05 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875218+0000", 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100259 2024-04-23T23:50:06.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3344 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6161108290000001, 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044300880000000004, 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3344 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871022+0000", 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3499999999999996e-07 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871025+0000", 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3199999999999998e-06 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871031+0000", 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4760000000000003e-06 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871036+0000", 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6530000000000002e-06 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875268+0000", 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042325890000000001 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875356+0000", 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8016999999999994e-05 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875452+0000", 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5497999999999999e-05 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3345 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6160867230000004, 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004594992, 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3345 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871046+0000", 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871048+0000", 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.995e-06 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871054+0000", 2024-04-23T23:50:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4890000000000003e-06 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871059+0000", 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8799999999999999e-06 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875479+0000", 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044206230000000003 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875551+0000", 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2089999999999996e-05 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875641+0000", 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9491000000000001e-05 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3346 3.a 3:572f4d31:::rbd_data.11798eaf3f6b.0000000000000007:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.871261+0000", 2024-04-23T23:50:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.615871361, 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045777150000000004, 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3346 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871261+0000", 2024-04-23T23:50:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871261+0000", 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871262+0000", 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9299999999999997e-07 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871266+0000", 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5560000000000001e-06 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871278+0000", 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1588e-05 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.871286+0000", 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3019999999999995e-06 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875664+0000", 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043780820000000002 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875735+0000", 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0992000000000001e-05 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.875839+0000", 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103402 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3558 3.a 3:526f43e1:::rbd_data.1179c62058ea.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.953052+0000", 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5340806809999998, 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050928459999999998, 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3558 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.508 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953052+0000", 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953052+0000", 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953054+0000", 2024-04-23T23:50:06.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9910000000000001e-06 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953064+0000", 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0662999999999999e-05 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953081+0000", 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6381000000000001e-05 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953090+0000", 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3840000000000003e-06 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953159+0000", 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8855000000000004e-05 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953306+0000", 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000146775 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.953505+0000", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000198746 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.955142+0000", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001637709 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.958042+0000", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029001209999999999 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.958102+0000", 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9308000000000002e-05 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.958145+0000", 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2913000000000001e-05 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3607 3.a 3:526f43e1:::rbd_data.1179c62058ea.000000000000000d:head [stat,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.019920+0000", 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.467211979, 2024-04-23T23:50:06.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052255180000000002, 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3607 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019920+0000", 2024-04-23T23:50:06.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019920+0000", 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019922+0000", 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7120000000000001e-06 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019930+0000", 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3189999999999997e-06 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019953+0000", 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.019962+0000", 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5929999999999995e-06 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020033+0000", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0352999999999995e-05 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020116+0000", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3163000000000003e-05 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.020257+0000", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014111500000000001 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.021149+0000", 2024-04-23T23:50:06.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000891931 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025038+0000", 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038888759999999999 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025094+0000", 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5826999999999999e-05 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.025146+0000", 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2410000000000001e-05 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3849 3.a 3:50bb8fc8:::rbd_data.1179c62058ea.0000000000000022:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.444425+0000", 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.0427069810000003, 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046949089999999997, 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3849 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444425+0000", 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444425+0000", 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444426+0000", 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3900000000000003e-07 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444431+0000", 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1799999999999998e-06 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444445+0000", 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4419999999999999e-05 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444452+0000", 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4329999999999998e-06 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444516+0000", 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3076000000000004e-05 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444600+0000", 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4148999999999997e-05 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444769+0000", 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.445664+0000", 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00089460600000000005 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449052+0000", 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003388622 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449082+0000", 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9904999999999999e-05 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449120+0000", 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8071000000000002e-05 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3850 3.10 3:09e2f0c1:::rbd_data.1179c62058ea.0000000000000026:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.444472+0000", 2024-04-23T23:50:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.042660283, 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045341360000000002, 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3850 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444472+0000", 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444472+0000", 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444473+0000", 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7199999999999999e-07 2024-04-23T23:50:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444476+0000", 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.875e-06 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444486+0000", 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-05 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444493+0000", 2024-04-23T23:50:06.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6329999999999999e-06 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444540+0000", 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7089000000000003e-05 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444623+0000", 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3634999999999995e-05 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.444790+0000", 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000166489 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.446121+0000", 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013310220000000001 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.448903+0000", 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027820810000000001 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.448956+0000", 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2580000000000001e-05 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.449006+0000", 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0729999999999997e-05 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:4857 3.a 3:57619caa:::rbd_data.1179c4f11774.000000000000000f:head [stat,write 3014656~10856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.799689+0000", 2024-04-23T23:50:06.530 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.6874429900000001, 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039895010000000003, 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4857 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799689+0000", 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799689+0000", 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799691+0000", 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5680000000000001e-06 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799709+0000", 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8121000000000001e-05 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799725+0000", 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5784e-05 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799733+0000", 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3180000000000007e-06 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799789+0000", 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.613e-05 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799853+0000", 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3596000000000006e-05 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.799940+0000", 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6590999999999998e-05 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.800680+0000", 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074060300000000005 2024-04-23T23:50:06.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.803623+0000", 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029428060000000001 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.803653+0000", 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0443999999999999e-05 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.803679+0000", 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5539999999999999e-05 2024-04-23T23:50:06.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5348 3.a 3:51876ddd:::rbd_data.1179c4f11774.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.945890+0000", 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.541242945, 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005120655, 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.536 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5348 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945890+0000", 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945890+0000", 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945891+0000", 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.615e-06 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945897+0000", 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945913+0000", 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5316e-05 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945921+0000", 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7630000000000006e-06 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.945993+0000", 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1143000000000006e-05 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.946000+0000", 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.396e-06 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947587+0000", 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015866999999999999 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947692+0000", 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105004 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.947809+0000", 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117612 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.948974+0000", 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011645150000000001 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950952+0000", 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019787030000000001 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.950985+0000", 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2119000000000002e-05 2024-04-23T23:50:06.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.951010+0000", 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5605e-05 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:5669 3.10 3:0bafcb9f:::rbd_data.1179c4f11774.0000000000000027:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:03.363618+0000", 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.123514251, 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040779429999999997, 2024-04-23T23:50:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 5669 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.363618+0000", 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.363618+0000", 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.363619+0000", 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9500000000000002e-07 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.363622+0000", 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.723e-06 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.363629+0000", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6750000000000002e-06 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.363635+0000", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3820000000000003e-06 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.364152+0000", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00051723600000000002 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.364217+0000", 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5389999999999996e-05 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.364311+0000", 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3718e-05 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.365336+0000", 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001024488 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.367630+0000", 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002294403 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.367659+0000", 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9023e-05 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:03.367696+0000", 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7209999999999998e-05 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:6520 3.10 3:0c3e62e9:::rbd_data.117967c411b0.000000000000002d:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.015264+0000", 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.47186827999999997, 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003739062, 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 6520 2024-04-23T23:50:06.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.015264+0000", 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.015264+0000", 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.015265+0000", 2024-04-23T23:50:06.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.34e-07 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.015267+0000", 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.12e-06 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.015272+0000", 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4190000000000001e-06 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.015277+0000", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4909999999999999e-06 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.016014+0000", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00073779300000000004 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.016130+0000", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115344 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019003+0000", 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028734609999999999 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:6551 3.10 3:0fef05e0:::rbd_data.117967c411b0.0000000000000027:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.018391+0000", 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.46874110899999999, 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003601234, 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 6551 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018391+0000", 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018391+0000", 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018392+0000", 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1900000000000001e-07 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018398+0000", 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2390000000000002e-06 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018410+0000", 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018423+0000", 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2771e-05 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018495+0000", 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2125999999999994e-05 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018573+0000", 2024-04-23T23:50:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8366000000000002e-05 2024-04-23T23:50:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018766+0000", 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000192079 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019918+0000", 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011519410000000001 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.021936+0000", 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020183470000000002 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.021968+0000", 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2345000000000003e-05 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.021993+0000", 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4389999999999999e-05 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:6552 3.10 3:0fe05222:::rbd_data.117967c411b0.000000000000002b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.018553+0000", 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.46857942699999999, 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062570569999999999, 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 6552 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018553+0000", 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018553+0000", 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018554+0000", 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7100000000000011e-07 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018559+0000", 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3809999999999998e-06 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018577+0000", 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7469000000000002e-05 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.018585+0000", 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2230000000000007e-06 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019032+0000", 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044658300000000002 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019083+0000", 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0952999999999998e-05 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019212+0000", 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129486 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.020588+0000", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013761380000000001 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024758+0000", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041699520000000002 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024788+0000", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9969000000000001e-05 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024810+0000", 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1931999999999999e-05 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:6557 3.10 3:0c3e62e9:::rbd_data.117967c411b0.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.019409+0000", 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.46772360200000002, 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055313960000000001, 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 6557 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019409+0000", 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019409+0000", 2024-04-23T23:50:06.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019410+0000", 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4799999999999998e-07 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019415+0000", 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3600000000000004e-06 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019424+0000", 2024-04-23T23:50:06.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9910000000000008e-06 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019430+0000", 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1360000000000002e-06 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019466+0000", 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6307999999999999e-05 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019511+0000", 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5151000000000002e-05 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.019678+0000", 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016692500000000001 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.020818+0000", 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001139372 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024850+0000", 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040322470000000001 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024894+0000", 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3995000000000001e-05 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024940+0000", 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6263000000000001e-05 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:6567 3.a 3:5506b24f:::rbd_data.117967c411b0.0000000000000036:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.024923+0000", 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.46220968600000001, 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035910400000000002, 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 6567 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024923+0000", 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024923+0000", 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024924+0000", 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3600000000000002e-07 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024929+0000", 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5829999999999997e-06 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024942+0000", 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3095000000000001e-05 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024951+0000", 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0280000000000006e-06 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.024990+0000", 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8909000000000003e-05 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.025061+0000", 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0751000000000001e-05 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028514+0000", 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034528380000000002 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:6577 3.10 3:0f50a52f:::rbd_data.117967c411b0.0000000000000039:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.028776+0000", 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.45835632199999998, 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042094929999999999, 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 6577 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028776+0000", 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028776+0000", 2024-04-23T23:50:06.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028777+0000", 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6899999999999998e-07 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028779+0000", 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028785+0000", 2024-04-23T23:50:06.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028789+0000", 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7160000000000001e-06 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028822+0000", 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2802999999999997e-05 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028879+0000", 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6915e-05 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.028989+0000", 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110327 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.031441+0000", 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002452348 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.032895+0000", 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001453141 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.032949+0000", 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4571000000000003e-05 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.032986+0000", 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6489999999999998e-05 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.580 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.581 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:06.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:06.583 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:06.596+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:06.586 INFO:teuthology.orchestra.run.smithi092.stdout:982 trunc from 0x38a1d77 to 0xed8ee79 2024-04-23T23:50:06.586 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:985 read 0xb138bfd thru 0xb147262 (0xe666 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:987 punch from 0x3854dcb to 0x38606c8, (0xb8fd bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:988 write 0x664d6e7 thru 0x665a0eb (0xca05 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:990 read 0x25ebf74 thru 0x25f8e1c (0xcea9 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:991 write 0x7a1def8 thru 0x7a20599 (0x26a2 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:992 punch from 0xa587796 to 0xa58f244, (0x7aae bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:996 write 0x765bc0f thru 0x7668902 (0xccf4 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:997 read 0xd2c2aa5 thru 0xd2d0ecc (0xe428 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:998 write 0x3b0db49 thru 0x3b14b71 (0x7029 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:1000 read 0xc4999a8 thru 0xc499e55 (0x4ae bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:1001 read 0x5088a59 thru 0x5089364 (0x90c bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:1002 write 0x853d7b thru 0x855da1 (0x2027 bytes) 2024-04-23T23:50:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:1003 writesame 0x5e32593 thru 0x5e3c2f6 data_size 0xe(0x9d64 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1004 writesame 0xaac98c2 thru 0xaacb6d2 data_size 0x1(0x1e11 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1005 read 0x51031e1 thru 0x510b5d0 (0x83f0 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1008 trunc from 0xed8ee79 to 0xdd45c61 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1009 read 0xbacc241 thru 0xbad6923 (0xa6e3 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1010 read 0x4fc1e74 thru 0x4fc23ff (0x58c bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1011 read 0x203c2bc thru 0x2045426 (0x916b bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1012 write 0x8ebb85c thru 0x8ec3790 (0x7f35 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1014 read 0x2258f59 thru 0x2264fc1 (0xc069 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1016 read 0x9af410a thru 0x9af72e6 (0x31dd bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1017 write 0xc32c0ae thru 0xc3367b9 (0xa70c bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1022 trunc from 0xdd45c61 to 0xdbca4de 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1023 punch from 0xda5d0c7 to 0xda6a867, (0xd7a0 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1025 write 0x5d04ff8 thru 0x5d0600d (0x1016 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1027 read 0xd18a4fe thru 0xd194baf (0xa6b2 bytes) 2024-04-23T23:50:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:1028 punch from 0x3b51315 to 0x3b5649d, (0x5188 bytes) 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1030 writesame 0xd5911bb thru 0xd596cc1 data_size 0x1(0x5b07 bytes) 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1032 trunc from 0xdbca4de to 0xabccf27 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1033 read 0x33fe8d0 thru 0x340a2af (0xb9e0 bytes) 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1036 punch from 0x187840d to 0x188686b, (0xe45e bytes) 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1037 trunc from 0xabccf27 to 0x4503fe2 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1039 trunc from 0x4503fe2 to 0x596147a 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1040 write 0x28d4c8d thru 0x28d512a (0x49e bytes) 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1042 read 0xe671a1 thru 0xe7390f (0xc76f bytes) 2024-04-23T23:50:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:1043 read 0x52af206 thru 0x52bde4c (0xec47 bytes) 2024-04-23T23:50:06.590 INFO:teuthology.orchestra.run.smithi092.stdout:1044 writesame 0x834357c thru 0x83480f6 data_size 0x39(0x4b7b bytes) 2024-04-23T23:50:06.590 INFO:teuthology.orchestra.run.smithi092.stdout:1045 writesame 0x3815e1a thru 0x38170f3 data_size 0x2(0x12da bytes) 2024-04-23T23:50:06.595 INFO:teuthology.orchestra.run.smithi092.stdout:1046 read 0x5eea1dd thru 0x5eedecf (0x3cf3 bytes) 2024-04-23T23:50:06.595 INFO:teuthology.orchestra.run.smithi092.stdout:1049 writesame 0x2b616f1 thru 0x2b62e19 data_size 0x7(0x1729 bytes) 2024-04-23T23:50:06.597 INFO:teuthology.orchestra.run.smithi092.stdout:1050 read 0x25d0ff5 thru 0x25d9335 (0x8341 bytes) 2024-04-23T23:50:06.598 INFO:teuthology.orchestra.run.smithi092.stdout:1053 writesame 0x4250073 thru 0x4250e4a data_size 0x4(0xdd8 bytes) 2024-04-23T23:50:06.608 INFO:teuthology.orchestra.run.smithi092.stdout:1055 write 0xb51afb6 thru 0xb529663 (0xe6ae bytes) 2024-04-23T23:50:06.612 INFO:teuthology.orchestra.run.smithi092.stdout:1056 trunc from 0xb529664 to 0xb426051 2024-04-23T23:50:06.617 INFO:teuthology.orchestra.run.smithi092.stdout:1057 read 0x5d9886b thru 0x5d98bf0 (0x386 bytes) 2024-04-23T23:50:06.617 INFO:teuthology.orchestra.run.smithi092.stdout:1058 clone 10 order 22 su 16384 sc 14 2024-04-23T23:50:06.683 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:06.697+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:06.726 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:06.726 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:06.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:06.783 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:06.797+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:06.836 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:06.836 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:06.836 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:06.837 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:06.837 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:06.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:06.884 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:06.898+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:06.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:06.951 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.763122959, 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:06.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:06.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:06.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3513 3.f 3:f2010bc9:::rbd_data.11798eaf3f6b.0000000000000000:8 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.943160+0000", 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0206998130000002, 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016151592999999999, 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3513 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943160+0000", 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943160+0000", 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943160+0000", 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9799999999999999e-07 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943161+0000", 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1939999999999999e-06 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943164+0000", 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9340000000000002e-06 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943168+0000", 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7610000000000001e-06 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959125+0000", 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015956884000000001 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959211+0000", 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6082999999999996e-05 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959311+0000", 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010043900000000001 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:3514 3.f 3:f2010bc9:::rbd_data.11798eaf3f6b.0000000000000000:8 [sparse-read 11010048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:01.943171+0000", 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.020688453, 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016295397999999999, 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3514 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943171+0000", 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943171+0000", 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943171+0000", 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0800000000000001e-07 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943172+0000", 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5199999999999995e-07 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943175+0000", 2024-04-23T23:50:06.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9409999999999999e-06 2024-04-23T23:50:06.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.943177+0000", 2024-04-23T23:50:06.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4090000000000001e-06 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959362+0000", 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016184135999999998 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959411+0000", 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9128000000000001e-05 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:01.959466+0000", 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5624000000000001e-05 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7017 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313072+0000", 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65078755700000002, 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016154135999999999, 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7017 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313072+0000", 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313072+0000", 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313072+0000", 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313073+0000", 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.302e-06 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313079+0000", 2024-04-23T23:50:06.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.237e-06 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313082+0000", 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7079999999999999e-06 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329216+0000", 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016133655 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329226+0000", 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9059999999999997e-06 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7018 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313089+0000", 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65077043800000001, 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016164582, 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7018 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313089+0000", 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313089+0000", 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313089+0000", 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3099999999999999e-07 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313091+0000", 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0600000000000002e-06 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313097+0000", 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5140000000000001e-06 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313101+0000", 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.715e-06 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329245+0000", 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016144496000000001 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329253+0000", 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4659999999999999e-06 2024-04-23T23:50:06.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7019 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313107+0000", 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65075191899999996, 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016184904, 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7019 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313107+0000", 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313107+0000", 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313108+0000", 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313109+0000", 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3379999999999999e-06 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313115+0000", 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4999999999999999e-06 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313118+0000", 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7840000000000001e-06 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329284+0000", 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016165149 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329292+0000", 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7830000000000004e-06 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.978 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7020 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313126+0000", 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65073352900000003, 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016204756000000001, 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.979 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7020 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313126+0000", 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313126+0000", 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313126+0000", 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4900000000000001e-07 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313127+0000", 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.342e-06 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313133+0000", 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3680000000000001e-06 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313137+0000", 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.827e-06 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329323+0000", 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016185858000000001 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329331+0000", 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0120000000000005e-06 2024-04-23T23:50:06.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7021 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313148+0000", 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65071118400000005, 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016220656999999999, 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7021 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313148+0000", 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313148+0000", 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313148+0000", 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313150+0000", 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.398e-06 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313155+0000", 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313159+0000", 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.771e-06 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329360+0000", 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016201073999999999 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329369+0000", 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5310000000000001e-06 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7024 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313166+0000", 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65069342900000005, 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016239269000000001, 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7024 2024-04-23T23:50:06.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313166+0000", 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313166+0000", 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313166+0000", 2024-04-23T23:50:06.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6399999999999998e-07 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313168+0000", 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.322e-06 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313173+0000", 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5609999999999998e-06 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313177+0000", 2024-04-23T23:50:06.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.693e-06 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329398+0000", 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016220752000000001 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329405+0000", 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5769999999999996e-06 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7025 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313183+0000", 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65067588899999995, 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016260237, 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7025 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313183+0000", 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313183+0000", 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313184+0000", 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4499999999999998e-07 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313185+0000", 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.33e-06 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313191+0000", 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.553e-06 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313194+0000", 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7179999999999998e-06 2024-04-23T23:50:06.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329435+0000", 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016240597999999998 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329444+0000", 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6929999999999996e-06 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7026 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313201+0000", 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65065820500000005, 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016266826000000002, 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7026 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:06.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313201+0000", 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313201+0000", 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313201+0000", 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2500000000000001e-07 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:06.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313203+0000", 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9910000000000001e-06 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313209+0000", 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6829999999999996e-06 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313213+0000", 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9920000000000001e-06 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329460+0000", 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016247214999999999 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329468+0000", 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6199999999999999e-06 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7028 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313220+0000", 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65063951200000003, 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016289914999999999, 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7028 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:06.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313220+0000", 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313220+0000", 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313220+0000", 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313232+0000", 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1549000000000001e-05 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313238+0000", 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2210000000000002e-06 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313242+0000", 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0430000000000001e-06 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329501+0000", 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016258763999999998 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329510+0000", 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0019999999999995e-06 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7029 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313249+0000", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65061020400000003, 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016297942999999999, 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7029 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313249+0000", 2024-04-23T23:50:07.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313249+0000", 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313249+0000", 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6899999999999998e-07 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313251+0000", 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3850000000000001e-06 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313257+0000", 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6579999999999999e-06 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313260+0000", 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7680000000000002e-06 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329539+0000", 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016278998999999999 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329547+0000", 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7640000000000005e-06 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.006 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7030 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313267+0000", 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65059254099999997, 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016324669999999999, 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7030 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313267+0000", 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313267+0000", 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313267+0000", 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7e-07 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313269+0000", 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.375e-06 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313274+0000", 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5799999999999999e-06 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313278+0000", 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9729999999999999e-06 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329583+0000", 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01630477 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329591+0000", 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.602e-06 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.010 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7031 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313285+0000", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65057473300000002, 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016344596999999999, 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7031 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313285+0000", 2024-04-23T23:50:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313285+0000", 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313285+0000", 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313286+0000", 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.39e-06 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313292+0000", 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4870000000000002e-06 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313296+0000", 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7009999999999998e-06 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329621+0000", 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016325922999999999 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329629+0000", 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7370000000000006e-06 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7032 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.014 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313302+0000", 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65055696200000002, 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016364927000000001, 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7032 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313302+0000", 2024-04-23T23:50:07.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313302+0000", 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313303+0000", 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3700000000000001e-07 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313304+0000", 2024-04-23T23:50:07.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.522e-06 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313310+0000", 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6910000000000002e-06 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313314+0000", 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0099999999999997e-06 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329659+0000", 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016344957 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329667+0000", 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4100000000000008e-06 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7035 3.11 3.5f5267f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313320+0000", 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65053882500000004, 2024-04-23T23:50:07.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016385931999999999, 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7035 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313320+0000", 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313320+0000", 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313321+0000", 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1600000000000002e-07 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313322+0000", 2024-04-23T23:50:07.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.345e-06 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313328+0000", 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313331+0000", 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329698+0000", 2024-04-23T23:50:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016366782999999999 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329706+0000", 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2199999999999992e-06 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7036 3.11 3.ae4a2611 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.313338+0000", 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65052147800000004, 2024-04-23T23:50:07.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016409252999999999, 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7036 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313338+0000", 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313338+0000", 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313338+0000", 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1399999999999998e-07 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313340+0000", 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.367e-06 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313345+0000", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.313349+0000", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6049999999999998e-06 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329739+0000", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016389945 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.329747+0000", 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.473e-06 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7612 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4980736~21233664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.338481+0000", 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.62537795900000004, 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.025701017, 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:07.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7612 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.338481+0000", 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.338481+0000", 2024-04-23T23:50:07.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.338483+0000", 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.218e-06 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.345248+0000", 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067649889999999999 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.345273+0000", 2024-04-23T23:50:07.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5928e-05 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.345285+0000", 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1377e-05 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.345361+0000", 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5735999999999995e-05 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.345471+0000", 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011006 2024-04-23T23:50:07.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.345913+0000", 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044261799999999998 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.364082+0000", 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018168304 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.364099+0000", 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7507e-05 2024-04-23T23:50:07.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.364139+0000", 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9782e-05 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.364182+0000", 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3498000000000003e-05 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:7613 3.16 e36/22 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:head v 36'36, mlcod=34'34)", 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.358941+0000", 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.60491815000000004, 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018898030999999999, 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.358941+0000", 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.358941+0000", 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.358942+0000", 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.136e-06 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.361191+0000", 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022482719999999999 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.361214+0000", 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3388999999999999e-05 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.361229+0000", 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4714e-05 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.361296+0000", 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7581000000000002e-05 2024-04-23T23:50:07.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.361324+0000", 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7647000000000001e-05 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.377789+0000", 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.016465227999999998 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.377839+0000", 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0064e-05 2024-04-23T23:50:07.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:07.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:07.036 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:07.037 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone9 from 0xb426051 (overlap 0x17c6665) to 0xe43d6f 2024-04-23T23:50:07.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:07.038 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:07.051+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:07.096 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-04-23T23:50:07.138 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:07.151+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:07.159 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:07.160 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:07.160 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:07.160 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:07.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:07.211 INFO:teuthology.orchestra.run.smithi092.stdout:1059 read 0x3d74401 thru 0x3d83a5b (0xf65b bytes) 2024-04-23T23:50:07.212 INFO:teuthology.orchestra.run.smithi092.stdout:1062 read 0x7f87c69 thru 0x7f8bdaf (0x4147 bytes) 2024-04-23T23:50:07.213 INFO:teuthology.orchestra.run.smithi092.stdout:1063 write 0x70fc29a thru 0x71037ab (0x7512 bytes) 2024-04-23T23:50:07.238 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:07.252+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:07.243 INFO:teuthology.orchestra.run.smithi092.stdout:1064 punch from 0x4964cb to 0x49cbfa, (0x672f bytes) 2024-04-23T23:50:07.243 INFO:teuthology.orchestra.run.smithi092.stdout:1066 write 0xb6d052 thru 0xb7c820 (0xf7cf bytes) 2024-04-23T23:50:07.301 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:07.302 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:07.302 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:07.302 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:07.302 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:07.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:07.343 INFO:teuthology.orchestra.run.smithi092.stdout:1067 read 0xb2cc62e thru 0xb2cd2ce (0xca1 bytes) 2024-04-23T23:50:07.343 INFO:teuthology.orchestra.run.smithi092.stdout:1068 read 0x616b134 thru 0x617973f (0xe60c bytes) 2024-04-23T23:50:07.344 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:07.357+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:07.344 INFO:teuthology.orchestra.run.smithi092.stdout:1069 writesame 0x9266da6 thru 0x926f082 data_size 0xd(0x82dd bytes) 2024-04-23T23:50:07.375 INFO:teuthology.orchestra.run.smithi092.stdout:1071 trunc from 0xb426051 to 0x98f901b 2024-04-23T23:50:07.440 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:07.454+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:07.479 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:07.479 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:07.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:07.479 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10283 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7225344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21860695699999999, 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035444584000000001, 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10283 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:07.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273274+0000", 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273279+0000", 2024-04-23T23:50:07.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5820000000000001e-06 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273284+0000", 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308551+0000", 2024-04-23T23:50:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035267169000000001 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308626+0000", 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4328999999999999e-05 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308716+0000", 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0647000000000001e-05 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10284 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7389184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21858892199999999, 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035618043000000002, 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10284 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273292+0000", 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9360000000000002e-06 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273298+0000", 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6860000000000003e-06 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273303+0000", 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1830000000000002e-06 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308748+0000", 2024-04-23T23:50:07.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035444749999999997 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308824+0000", 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6705999999999994e-05 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308908+0000", 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10285 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21856030400000001, 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035744949999999998, 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.492 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10285 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273319+0000", 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6000000000000004e-07 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273322+0000", 2024-04-23T23:50:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.236e-06 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273329+0000", 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1350000000000003e-06 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273335+0000", 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5149999999999997e-06 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308939+0000", 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035603856000000003 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309006+0000", 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6934999999999995e-05 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309063+0000", 2024-04-23T23:50:07.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7713000000000001e-05 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10286 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21853428499999999, 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035978035999999998, 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10286 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273345+0000", 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273347+0000", 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.125e-06 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273363+0000", 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6353000000000001e-05 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273370+0000", 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-04-23T23:50:07.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309119+0000", 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035748874 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309212+0000", 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2523999999999997e-05 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309322+0000", 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110597 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10287 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8142848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21849998100000001, 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036157685000000002, 2024-04-23T23:50:07.502 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10287 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273382+0000", 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4959999999999999e-06 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273389+0000", 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9160000000000004e-06 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273394+0000", 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1710000000000001e-06 2024-04-23T23:50:07.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309383+0000", 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035989128000000002 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309448+0000", 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5419999999999994e-05 2024-04-23T23:50:07.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309536+0000", 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8034000000000005e-05 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10288 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8306688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:07.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21846768499999999, 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036258020000000002, 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10288 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-04-23T23:50:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273414+0000", 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7259999999999998e-06 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273422+0000", 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5390000000000002e-06 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273427+0000", 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309563+0000", 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036135689999999998 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309618+0000", 2024-04-23T23:50:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4994000000000002e-05 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309669+0000", 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1147999999999999e-05 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10289 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8470528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21843451699999999, 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036365322999999998, 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10289 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273445+0000", 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273447+0000", 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.587e-06 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273456+0000", 2024-04-23T23:50:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6770000000000001e-06 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273462+0000", 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8810000000000001e-06 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309690+0000", 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036228306000000002 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309753+0000", 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2772000000000001e-05 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309809+0000", 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6560000000000001e-05 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10290 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.218409199, 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036521275999999998, 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10290 2024-04-23T23:50:07.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273470+0000", 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-07 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273472+0000", 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2330000000000001e-06 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273479+0000", 2024-04-23T23:50:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4080000000000003e-06 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273484+0000", 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9819999999999999e-06 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309838+0000", 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036354356999999997 2024-04-23T23:50:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309923+0000", 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5130000000000007e-05 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309991+0000", 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7630000000000001e-05 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10291 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9060352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21838611899999999, 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036628916999999997, 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10291 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0900000000000002e-07 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273495+0000", 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3190000000000002e-06 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273502+0000", 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4609999999999996e-06 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273507+0000", 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9490000000000004e-06 2024-04-23T23:50:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310011+0000", 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036504395000000002 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310057+0000", 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6008999999999999e-05 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310121+0000", 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4275000000000004e-05 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10292 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9224192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.218334478, 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036703116000000001, 2024-04-23T23:50:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10292 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273545+0000", 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273548+0000", 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273557+0000", 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2979999999999997e-06 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273563+0000", 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7810000000000002e-06 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310148+0000", 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036585212999999998 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310198+0000", 2024-04-23T23:50:07.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0686999999999998e-05 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310247+0000", 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8801000000000003e-05 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10293 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9388032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.218227687, 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036741629999999997, 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10293 2024-04-23T23:50:07.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273652+0000", 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273665+0000", 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.34e-05 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273673+0000", 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3070000000000003e-06 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273679+0000", 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8610000000000003e-06 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310264+0000", 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036584942000000002 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310304+0000", 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9932999999999999e-05 2024-04-23T23:50:07.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310393+0000", 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8640999999999994e-05 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10294 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9551872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.539 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21819101900000001, 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036864814000000003, 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10294 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273690+0000", 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8139999999999999e-06 2024-04-23T23:50:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273696+0000", 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7760000000000003e-06 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273701+0000", 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0159999999999999e-06 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310423+0000", 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036722312999999999 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310489+0000", 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5741000000000006e-05 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310552+0000", 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3674000000000001e-05 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10295 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9977856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21817076899999999, 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036991522999999998, 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.545 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10295 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:07.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.77e-07 2024-04-23T23:50:07.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273710+0000", 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0829999999999998e-06 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273716+0000", 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9410000000000004e-06 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273721+0000", 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.955e-06 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310584+0000", 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036862930000000002 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310644+0000", 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0037000000000001e-05 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310699+0000", 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.52e-05 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10296 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10141696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.218148908, 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037192338999999998, 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10296 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:07.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273732+0000", 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273739+0000", 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5599999999999999e-06 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273744+0000", 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0980000000000001e-06 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310736+0000", 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036992522999999999 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310811+0000", 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4184999999999995e-05 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310922+0000", 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111532 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10297 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10305536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21811778000000001, 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037380582000000002, 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10297 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273764+0000", 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273790+0000", 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273797+0000", 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0129999999999996e-06 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310950+0000", 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037152875000000002 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311040+0000", 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9816999999999997e-05 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311141+0000", 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010139700000000001 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10298 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10469376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21807253900000001, 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037503460000000002, 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10298 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273807+0000", 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8800000000000003e-07 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273809+0000", 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273816+0000", 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5830000000000003e-06 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273821+0000", 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.848e-06 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311188+0000", 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037366982 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311234+0000", 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6269e-05 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311310+0000", 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780999999999998e-05 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10299 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10895360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21805143900000001, 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037660666000000002, 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10299 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273828+0000", 2024-04-23T23:50:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273835+0000", 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273844+0000", 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1360000000000003e-06 2024-04-23T23:50:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273850+0000", 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3360000000000002e-06 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311345+0000", 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037495406000000002 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311415+0000", 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9605999999999997e-05 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311488+0000", 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3161e-05 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10300 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11059200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21798787, 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037811175000000002, 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10300 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2200000000000004e-07 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273894+0000", 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9550000000000001e-06 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273903+0000", 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3299999999999999e-06 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273910+0000", 2024-04-23T23:50:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8689999999999998e-06 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311574+0000", 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037664133000000002 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311616+0000", 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2481000000000002e-05 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311702+0000", 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5785000000000006e-05 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10301 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11223040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21796074500000001, 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037936401000000002, 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10301 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.44e-07 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273921+0000", 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273927+0000", 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5270000000000004e-06 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273932+0000", 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.215e-06 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311737+0000", 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037804799 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311801+0000", 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3652999999999995e-05 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311854+0000", 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3542999999999999e-05 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10302 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11386880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.21793758999999999, 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038084931000000002, 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10302 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273944+0000", 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273950+0000", 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6540000000000002e-06 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273956+0000", 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311874+0000", 2024-04-23T23:50:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037918232000000003 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311951+0000", 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6701999999999998e-05 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.312026+0000", 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5469000000000005e-05 2024-04-23T23:50:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:07.591 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:07.592 INFO:teuthology.orchestra.run.smithi092.stdout:1072 punch from 0x9860438 to 0x9866881, (0x6449 bytes) 2024-04-23T23:50:07.592 INFO:teuthology.orchestra.run.smithi092.stdout:1075 read 0x369b95f thru 0x36ab1e8 (0xf88a bytes) 2024-04-23T23:50:07.592 INFO:teuthology.orchestra.run.smithi092.stdout:1077 trunc from 0x98f901b to 0x82ed611 2024-04-23T23:50:07.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:07.594 INFO:teuthology.orchestra.run.smithi092.stdout:1080 punch from 0x4c0690c to 0x4c0c1c8, (0x58bc bytes) 2024-04-23T23:50:07.594 INFO:teuthology.orchestra.run.smithi092.stdout:1081 trunc from 0x82ed611 to 0x1b37fd8 2024-04-23T23:50:07.594 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:07.606+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:07.693 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:07.707+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:07.699 INFO:teuthology.orchestra.run.smithi092.stdout:1082 write 0x7d50295 thru 0x7d5cc06 (0xc972 bytes) 2024-04-23T23:50:07.706 INFO:teuthology.orchestra.run.smithi092.stdout:1083 punch from 0x1c8fb62 to 0x1c9723b, (0x76d9 bytes) 2024-04-23T23:50:07.706 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:07.706 INFO:teuthology.orchestra.run.smithi092.stdout:1086 read 0x19c391c thru 0x19c5c51 (0x2336 bytes) 2024-04-23T23:50:07.707 INFO:teuthology.orchestra.run.smithi092.stdout:1087 writesame 0x70096a0 thru 0x70112c9 data_size 0x17(0x7c2a bytes) 2024-04-23T23:50:07.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:07.715 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:07.715 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:07.715 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:07.715 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:07.715 INFO:teuthology.orchestra.run.smithi092.stdout:1093 clone 11 order 24 su 131072 sc 3 2024-04-23T23:50:07.747 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:07.747 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:07.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:07.793 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:07.807+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:07.836 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:07.836 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:07.853 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:07.854 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:07.871 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:07.888 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:07.888 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:07.894 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:07.908+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:07.903 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:07.903 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:07.905 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:07.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:07.923 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:07.941 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:07.941 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:07.958 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:50:07.961 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:07.995 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:08.009+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:08.029 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone10 2024-04-23T23:50:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:08.096 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:08.109+0000 7f8f8ee44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32509) UID: 0 2024-04-23T23:50:08.143 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-04-23T23:50:08.162 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.162 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:08.182 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.185 INFO:teuthology.orchestra.run.smithi092.stdout:1096 trunc from 0x7d5cc07 to 0xa9f86ab 2024-04-23T23:50:08.194 INFO:teuthology.orchestra.run.smithi092.stdout:1097 read 0x2dcb6b4 thru 0x2dce7a9 (0x30f6 bytes) 2024-04-23T23:50:08.194 INFO:teuthology.orchestra.run.smithi092.stdout:1098 write 0x3a4c5ec thru 0x3a4dd63 (0x1778 bytes) 2024-04-23T23:50:08.197 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:08.211+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:08.203 INFO:teuthology.orchestra.run.smithi092.stdout:1099 write 0x13db93e thru 0x13e67a1 (0xae64 bytes) 2024-04-23T23:50:08.298 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:08.311+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:08.328 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.328 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:08.368 INFO:teuthology.orchestra.run.smithi092.stdout:1100 punch from 0x82f7339 to 0x82fee3c, (0x7b03 bytes) 2024-04-23T23:50:08.368 INFO:teuthology.orchestra.run.smithi092.stdout:1101 write 0x4770636 thru 0x4772ab1 (0x247c bytes) 2024-04-23T23:50:08.370 INFO:teuthology.orchestra.run.smithi092.stdout:1102 writesame 0x2f33f3c thru 0x2f3eea4 data_size 0x7(0xaf69 bytes) 2024-04-23T23:50:08.374 INFO:teuthology.orchestra.run.smithi092.stdout:1103 read 0x634517b thru 0x634eec8 (0x9d4e bytes) 2024-04-23T23:50:08.374 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.374 INFO:teuthology.orchestra.run.smithi092.stdout:1106 writesame 0x46f8e thru 0x4f7e5 data_size 0x4(0x8858 bytes) 2024-04-23T23:50:08.381 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.381 INFO:teuthology.orchestra.run.smithi092.stdout:1110 punch from 0x4bec6f5 to 0x4bfc202, (0xfb0d bytes) 2024-04-23T23:50:08.381 INFO:teuthology.orchestra.run.smithi092.stdout:1111 punch from 0xa4ae811 to 0xa4b6b8e, (0x837d bytes) 2024-04-23T23:50:08.381 INFO:teuthology.orchestra.run.smithi092.stdout:1112 punch from 0x595fcf5 to 0x5968c72, (0x8f7d bytes) 2024-04-23T23:50:08.381 INFO:teuthology.orchestra.run.smithi092.stdout:1113 read 0x88d5f4d thru 0x88e17bc (0xb870 bytes) 2024-04-23T23:50:08.385 INFO:teuthology.orchestra.run.smithi092.stdout:1114 trunc from 0xa9f86ab to 0x6dc200a 2024-04-23T23:50:08.399 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:08.412+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:08.418 INFO:teuthology.orchestra.run.smithi092.stdout:1115 read 0x18066af thru 0x1808cce (0x2620 bytes) 2024-04-23T23:50:08.418 INFO:teuthology.orchestra.run.smithi092.stdout:1116 write 0x4707b5a thru 0x4710ec8 (0x936f bytes) 2024-04-23T23:50:08.427 INFO:teuthology.orchestra.run.smithi092.stdout:1117 trunc from 0x6dc200a to 0x151a459 2024-04-23T23:50:08.499 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:08.513+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:08.500 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.501 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:08.529 INFO:teuthology.orchestra.run.smithi092.stdout:1119 read 0xe061a7 thru 0xe0f612 (0x946c bytes) 2024-04-23T23:50:08.531 INFO:teuthology.orchestra.run.smithi092.stdout:1121 read 0x10b79b1 thru 0x10be16a (0x67ba bytes) 2024-04-23T23:50:08.531 INFO:teuthology.orchestra.run.smithi092.stdout:1122 read 0xb98fa5 thru 0xba3cef (0xad4b bytes) 2024-04-23T23:50:08.548 INFO:teuthology.orchestra.run.smithi092.stdout:1124 write 0xa170eb9 thru 0xa17e2bf (0xd407 bytes) 2024-04-23T23:50:08.551 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.551 INFO:teuthology.orchestra.run.smithi092.stdout:1127 punch from 0x5d015ce to 0x5d03592, (0x1fc4 bytes) 2024-04-23T23:50:08.562 INFO:teuthology.orchestra.run.smithi092.stdout:1129 trunc from 0xa17e2c0 to 0xd9c3e7 2024-04-23T23:50:08.575 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.593 INFO:teuthology.orchestra.run.smithi092.stdout:1133 write 0xb746a2b thru 0xb74ca32 (0x6008 bytes) 2024-04-23T23:50:08.597 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.597 INFO:teuthology.orchestra.run.smithi092.stdout:1135 writesame 0xa958c68 thru 0xa95f81d data_size 0x11(0x6bb6 bytes) 2024-04-23T23:50:08.603 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:08.614+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:08.603 INFO:teuthology.orchestra.run.smithi092.stdout:1136 read 0x5bc8897 thru 0x5bd33a8 (0xab12 bytes) 2024-04-23T23:50:08.603 INFO:teuthology.orchestra.run.smithi092.stdout:1138 write 0x46cfb59 thru 0x46d036c (0x814 bytes) 2024-04-23T23:50:08.603 INFO:teuthology.orchestra.run.smithi092.stdout:1139 write 0x59115f7 thru 0x5918a1a (0x7424 bytes) 2024-04-23T23:50:08.608 INFO:teuthology.orchestra.run.smithi092.stdout:1140 trunc from 0xb74ca33 to 0xdb3c912 2024-04-23T23:50:08.625 INFO:teuthology.orchestra.run.smithi092.stdout:1141 trunc from 0xdb3c912 to 0x3d31bda 2024-04-23T23:50:08.635 INFO:teuthology.orchestra.run.smithi092.stdout:1143 read 0x1653e3c thru 0x165e976 (0xab3b bytes) 2024-04-23T23:50:08.636 INFO:teuthology.orchestra.run.smithi092.stdout:1144 punch from 0x219676d to 0x219bc73, (0x5506 bytes) 2024-04-23T23:50:08.642 INFO:teuthology.orchestra.run.smithi092.stdout:1146 writesame 0x5047a9a thru 0x50548bc data_size 0x1(0xce23 bytes) 2024-04-23T23:50:08.642 INFO:teuthology.orchestra.run.smithi092.stdout:1147 write 0x3f99292 thru 0x3fa76f3 (0xe462 bytes) 2024-04-23T23:50:08.644 INFO:teuthology.orchestra.run.smithi092.stdout:1148 writesame 0x21f8d54 thru 0x21fc350 data_size 0x11(0x35fd bytes) 2024-04-23T23:50:08.653 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.653 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.656 INFO:teuthology.orchestra.run.smithi092.stdout:1149 write 0x9818f81 thru 0x982315d (0xa1dd bytes) 2024-04-23T23:50:08.661 INFO:teuthology.orchestra.run.smithi092.stdout:1152 read 0x426062 thru 0x431cfd (0xbc9c bytes) 2024-04-23T23:50:08.661 INFO:teuthology.orchestra.run.smithi092.stdout:1153 punch from 0x7db452a to 0x7db7157, (0x2c2d bytes) 2024-04-23T23:50:08.661 INFO:teuthology.orchestra.run.smithi092.stdout:1154 read 0x50e5833 thru 0x50e685e (0x102c bytes) 2024-04-23T23:50:08.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:08.671 INFO:teuthology.orchestra.run.smithi092.stdout:1158 writesame 0xc373828 thru 0xc37e372 data_size 0x3(0xab4b bytes) 2024-04-23T23:50:08.678 INFO:teuthology.orchestra.run.smithi092.stdout:1159 write 0xe4b2b1e thru 0xe4b69dc (0x3ebf bytes) 2024-04-23T23:50:08.682 INFO:teuthology.orchestra.run.smithi092.stdout:1160 read 0xeba574 thru 0xebc55e (0x1feb bytes) 2024-04-23T23:50:08.682 INFO:teuthology.orchestra.run.smithi092.stdout:1161 write 0xc22bcc7 thru 0xc23a35b (0xe695 bytes) 2024-04-23T23:50:08.684 INFO:teuthology.orchestra.run.smithi092.stdout:1162 writesame 0xd9b2b6 thru 0xda670b data_size 0x2(0xb456 bytes) 2024-04-23T23:50:08.701 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:08.715+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:08.702 INFO:teuthology.orchestra.run.smithi092.stdout:1163 writesame 0x51a898c thru 0x51b2508 data_size 0x13(0x9b7d bytes) 2024-04-23T23:50:08.705 INFO:teuthology.orchestra.run.smithi092.stdout:1164 write 0xa755c34 thru 0xa761f9a (0xc367 bytes) 2024-04-23T23:50:08.709 INFO:teuthology.orchestra.run.smithi092.stdout:1165 writesame 0x44ebc48 thru 0x44efbb9 data_size 0x3(0x3f72 bytes) 2024-04-23T23:50:08.713 INFO:teuthology.orchestra.run.smithi092.stdout:1166 write 0xa72a1f5 thru 0xa72c800 (0x260c bytes) 2024-04-23T23:50:08.715 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.715 INFO:teuthology.orchestra.run.smithi092.stdout:1170 write 0x85d84c thru 0x8675f6 (0x9dab bytes) 2024-04-23T23:50:08.717 INFO:teuthology.orchestra.run.smithi092.stdout:1171 trunc from 0xe4b69dd to 0xe8eeba0 2024-04-23T23:50:08.721 INFO:teuthology.orchestra.run.smithi092.stdout:1172 writesame 0x1d69fa thru 0x1e5409 data_size 0x8c(0xea10 bytes) 2024-04-23T23:50:08.723 INFO:teuthology.orchestra.run.smithi092.stdout:1173 read 0x41b4d7c thru 0x41bf3af (0xa634 bytes) 2024-04-23T23:50:08.723 INFO:teuthology.orchestra.run.smithi092.stdout:1174 writesame 0x71fb463 thru 0x720811c data_size 0xf(0xccba bytes) 2024-04-23T23:50:08.728 INFO:teuthology.orchestra.run.smithi092.stdout:1176 read 0x286b037 thru 0x286bb09 (0xad3 bytes) 2024-04-23T23:50:08.728 INFO:teuthology.orchestra.run.smithi092.stdout:1179 write 0x647d9a0 thru 0x647e475 (0xad6 bytes) 2024-04-23T23:50:08.733 INFO:teuthology.orchestra.run.smithi092.stdout:1180 trunc from 0xe8eeba0 to 0xba0c1f2 2024-04-23T23:50:08.748 INFO:teuthology.orchestra.run.smithi092.stdout:1181 punch from 0x743afc8 to 0x744a2e7, (0xf31f bytes) 2024-04-23T23:50:08.750 INFO:teuthology.orchestra.run.smithi092.stdout:1182 trunc from 0xba0c1f2 to 0xa05d3b3 2024-04-23T23:50:08.758 INFO:teuthology.orchestra.run.smithi092.stdout:1183 read 0xdbe7e5 thru 0xdce36b (0xfb87 bytes) 2024-04-23T23:50:08.758 INFO:teuthology.orchestra.run.smithi092.stdout:1184 read 0x6fa0a4c thru 0x6fa8e78 (0x842d bytes) 2024-04-23T23:50:08.758 INFO:teuthology.orchestra.run.smithi092.stdout:1185 write 0x828927f thru 0x828f9ec (0x676e bytes) 2024-04-23T23:50:08.769 INFO:teuthology.orchestra.run.smithi092.stdout:1186 write 0xd8a63d3 thru 0xd8b2e2b (0xca59 bytes) 2024-04-23T23:50:08.772 INFO:teuthology.orchestra.run.smithi092.stdout:1188 read 0x53d9e89 thru 0x53e7312 (0xd48a bytes) 2024-04-23T23:50:08.773 INFO:teuthology.orchestra.run.smithi092.stdout:1189 write 0xdf352f thru 0xdf8c00 (0x56d2 bytes) 2024-04-23T23:50:08.780 INFO:teuthology.orchestra.run.smithi092.stdout:1190 trunc from 0xd8b2e2c to 0x836448c 2024-04-23T23:50:08.784 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.784 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:08.799 INFO:teuthology.orchestra.run.smithi092.stdout:1191 write 0x931b0bc thru 0x931e312 (0x3257 bytes) 2024-04-23T23:50:08.802 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:08.815+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:08.809 INFO:teuthology.orchestra.run.smithi092.stdout:1193 writesame 0xac0f904 thru 0xac1f70e data_size 0x1(0xfe0b bytes) 2024-04-23T23:50:08.809 INFO:teuthology.orchestra.run.smithi092.stdout:1194 write 0x1a21f53 thru 0x1a2d597 (0xb645 bytes) 2024-04-23T23:50:08.810 INFO:teuthology.orchestra.run.smithi092.stdout:1195 read 0x211a10f thru 0x2120b89 (0x6a7b bytes) 2024-04-23T23:50:08.817 INFO:teuthology.orchestra.run.smithi092.stdout:1196 trunc from 0xac1f70f to 0x62584cd 2024-04-23T23:50:08.827 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.827 INFO:teuthology.orchestra.run.smithi092.stdout:1198 punch from 0x4345d9d to 0x4349dc3, (0x4026 bytes) 2024-04-23T23:50:08.838 INFO:teuthology.orchestra.run.smithi092.stdout:1199 writesame 0xb237e5f thru 0xb244b7c data_size 0x3b(0xcd1e bytes) 2024-04-23T23:50:08.845 INFO:teuthology.orchestra.run.smithi092.stdout:1200 trunc from 0xb244b7d to 0x59126d8 2024-04-23T23:50:08.856 INFO:teuthology.orchestra.run.smithi092.stdout:1202 read 0x36cff0c thru 0x36d1c5b (0x1d50 bytes) 2024-04-23T23:50:08.856 INFO:teuthology.orchestra.run.smithi092.stdout:1203 write 0x5025a5c thru 0x50289e8 (0x2f8d bytes) 2024-04-23T23:50:08.863 INFO:teuthology.orchestra.run.smithi092.stdout:1204 trunc from 0x59126d8 to 0x9f7e3da 2024-04-23T23:50:08.868 INFO:teuthology.orchestra.run.smithi092.stdout:1206 punch from 0x74469f4 to 0x744e98d, (0x7f99 bytes) 2024-04-23T23:50:08.868 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.868 INFO:teuthology.orchestra.run.smithi092.stdout:1209 punch from 0x8d892b7 to 0x8d913f0, (0x8139 bytes) 2024-04-23T23:50:08.868 INFO:teuthology.orchestra.run.smithi092.stdout:1211 punch from 0x5e0ec98 to 0x5e141ae, (0x5516 bytes) 2024-04-23T23:50:08.868 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.872 INFO:teuthology.orchestra.run.smithi092.stdout:1214 write 0x9f7dff6 thru 0x9f8651e (0x8529 bytes) 2024-04-23T23:50:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:1215 punch from 0x52b1ecb to 0x52b3dd2, (0x1f07 bytes) 2024-04-23T23:50:08.878 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.878 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.878 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.878 INFO:teuthology.orchestra.run.smithi092.stdout:1221 write 0x8a60314 thru 0x8a66152 (0x5e3f bytes) 2024-04-23T23:50:08.880 INFO:teuthology.orchestra.run.smithi092.stdout:1222 read 0x56fd3e4 thru 0x57076d7 (0xa2f4 bytes) 2024-04-23T23:50:08.887 INFO:teuthology.orchestra.run.smithi092.stdout:1224 write 0xb6cc45a thru 0xb6d170a (0x52b1 bytes) 2024-04-23T23:50:08.889 INFO:teuthology.orchestra.run.smithi092.stdout:1226 write 0x8b3c52d thru 0x8b4ae80 (0xe954 bytes) 2024-04-23T23:50:08.893 INFO:teuthology.orchestra.run.smithi092.stdout:1230 read 0x956d85b thru 0x957a237 (0xc9dd bytes) 2024-04-23T23:50:08.894 INFO:teuthology.orchestra.run.smithi092.stdout:1231 read 0x444b5d1 thru 0x444e734 (0x3164 bytes) 2024-04-23T23:50:08.894 INFO:teuthology.orchestra.run.smithi092.stdout:1232 read 0x6f0faa1 thru 0x6f1f70e (0xfc6e bytes) 2024-04-23T23:50:08.894 INFO:teuthology.orchestra.run.smithi092.stdout:1233 write 0xb1bf71f thru 0xb1c6a37 (0x7319 bytes) 2024-04-23T23:50:08.897 INFO:teuthology.orchestra.run.smithi092.stdout:1234 punch from 0x4fa137 to 0x501986, (0x784f bytes) 2024-04-23T23:50:08.897 INFO:teuthology.orchestra.run.smithi092.stdout:1235 write 0xa458af1 thru 0xa465c4e (0xd15e bytes) 2024-04-23T23:50:08.898 INFO:teuthology.orchestra.run.smithi092.stdout:1236 read 0x4b8b9e2 thru 0x4b99c73 (0xe292 bytes) 2024-04-23T23:50:08.899 INFO:teuthology.orchestra.run.smithi092.stdout:1238 writesame 0x5d6151d thru 0x5d6fde2 data_size 0x76(0xe8c6 bytes) 2024-04-23T23:50:08.902 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:08.902 INFO:teuthology.orchestra.run.smithi092.stdout:1242 read 0xac19b2 thru 0xac2a46 (0x1095 bytes) 2024-04-23T23:50:08.903 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:08.916+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:08.903 INFO:teuthology.orchestra.run.smithi092.stdout:1245 write 0x2c482ad thru 0x2c4de45 (0x5b99 bytes) 2024-04-23T23:50:08.903 INFO:teuthology.orchestra.run.smithi092.stdout:1246 writesame 0x83cc18d thru 0x83cd589 data_size 0x11(0x13fd bytes) 2024-04-23T23:50:08.906 INFO:teuthology.orchestra.run.smithi092.stdout:1247 punch from 0x38b2b5c to 0x38c1a44, (0xeee8 bytes) 2024-04-23T23:50:08.910 INFO:teuthology.orchestra.run.smithi092.stdout:1248 trunc from 0xb6d170b to 0x7cd1da3 2024-04-23T23:50:08.915 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:08.915 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:08.923 INFO:teuthology.orchestra.run.smithi092.stdout:1249 punch from 0x13669e6 to 0x1374673, (0xdc8d bytes) 2024-04-23T23:50:08.923 INFO:teuthology.orchestra.run.smithi092.stdout:1250 punch from 0x70b8dc2 to 0x70c4acc, (0xbd0a bytes) 2024-04-23T23:50:08.923 INFO:teuthology.orchestra.run.smithi092.stdout:1251 read 0x18c525c thru 0x18ce7a3 (0x9548 bytes) 2024-04-23T23:50:08.923 INFO:teuthology.orchestra.run.smithi092.stdout:1252 read 0x22c38 thru 0x25d07 (0x30d0 bytes) 2024-04-23T23:50:08.925 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:08.935 INFO:teuthology.orchestra.run.smithi092.stdout:1253 write 0xd1b2ee9 thru 0xd1bc22e (0x9346 bytes) 2024-04-23T23:50:08.937 INFO:teuthology.orchestra.run.smithi092.stdout:1255 writesame 0x68f9e30 thru 0x68feb61 data_size 0x29(0x4d32 bytes) 2024-04-23T23:50:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:1258 write 0x629ce82 thru 0x62a57b7 (0x8936 bytes) 2024-04-23T23:50:08.942 INFO:teuthology.orchestra.run.smithi092.stdout:1260 punch from 0xacebcf4 to 0xacf100f, (0x531b bytes) 2024-04-23T23:50:08.943 INFO:teuthology.orchestra.run.smithi092.stdout:1261 write 0x49da2c8 thru 0x49ddeae (0x3be7 bytes) 2024-04-23T23:50:08.944 INFO:teuthology.orchestra.run.smithi092.stdout:1262 punch from 0xaf9cb67 to 0xafa53df, (0x8878 bytes) 2024-04-23T23:50:08.944 INFO:teuthology.orchestra.run.smithi092.stdout:1263 read 0x16c4aa3 thru 0x16d0b34 (0xc092 bytes) 2024-04-23T23:50:08.944 INFO:teuthology.orchestra.run.smithi092.stdout:1264 writesame 0x7082122 thru 0x708935a data_size 0x39(0x7239 bytes) 2024-04-23T23:50:08.946 INFO:teuthology.orchestra.run.smithi092.stdout:1265 writesame 0xaf5194d thru 0xaf5daf7 data_size 0x1(0xc1ab bytes) 2024-04-23T23:50:08.986 INFO:teuthology.orchestra.run.smithi092.stdout:1267 trunc from 0xd1bc22f to 0x29747c0 2024-04-23T23:50:09.000 INFO:teuthology.orchestra.run.smithi092.stdout:1268 write 0x12e1b69 thru 0x12e3345 (0x17dd bytes) 2024-04-23T23:50:09.002 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.003 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:09.017+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:09.011 INFO:teuthology.orchestra.run.smithi092.stdout:1272 trunc from 0x29747c0 to 0xa22dcf8 2024-04-23T23:50:09.016 INFO:teuthology.orchestra.run.smithi092.stdout:1273 punch from 0x1ddf5a to 0x1e35a7, (0x564d bytes) 2024-04-23T23:50:09.016 INFO:teuthology.orchestra.run.smithi092.stdout:1276 write 0x5afb262 thru 0x5b07993 (0xc732 bytes) 2024-04-23T23:50:09.022 INFO:teuthology.orchestra.run.smithi092.stdout:1277 read 0x66a4b31 thru 0x66a5aa9 (0xf79 bytes) 2024-04-23T23:50:09.022 INFO:teuthology.orchestra.run.smithi092.stdout:1278 writesame 0x2186bfe thru 0x218b783 data_size 0x7(0x4b86 bytes) 2024-04-23T23:50:09.024 INFO:teuthology.orchestra.run.smithi092.stdout:1280 read 0x3ed00d5 thru 0x3ed2bf6 (0x2b22 bytes) 2024-04-23T23:50:09.025 INFO:teuthology.orchestra.run.smithi092.stdout:1281 writesame 0x736ebd0 thru 0x737e46c data_size 0x5(0xf89d bytes) 2024-04-23T23:50:09.033 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.033 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.041 INFO:teuthology.orchestra.run.smithi092.stdout:1282 write 0xbec16be thru 0xbeca76d (0x90b0 bytes) 2024-04-23T23:50:09.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:09.044 INFO:teuthology.orchestra.run.smithi092.stdout:1283 read 0xb256208 thru 0xb25a22c (0x4025 bytes) 2024-04-23T23:50:09.044 INFO:teuthology.orchestra.run.smithi092.stdout:1285 read 0x8a730fe thru 0x8a7e3f3 (0xb2f6 bytes) 2024-04-23T23:50:09.044 INFO:teuthology.orchestra.run.smithi092.stdout:1287 read 0x90e6ae thru 0x9180b2 (0x9a05 bytes) 2024-04-23T23:50:09.045 INFO:teuthology.orchestra.run.smithi092.stdout:1288 read 0x2ab292e thru 0x2ab4b7b (0x224e bytes) 2024-04-23T23:50:09.045 INFO:teuthology.orchestra.run.smithi092.stdout:1289 write 0x698240d thru 0x69884e2 (0x60d6 bytes) 2024-04-23T23:50:09.049 INFO:teuthology.orchestra.run.smithi092.stdout:1290 read 0x64d5520 thru 0x64e2e67 (0xd948 bytes) 2024-04-23T23:50:09.056 INFO:teuthology.orchestra.run.smithi092.stdout:1291 write 0xce592ec thru 0xce5c09b (0x2db0 bytes) 2024-04-23T23:50:09.060 INFO:teuthology.orchestra.run.smithi092.stdout:1293 punch from 0x592c8b8 to 0x592f3d9, (0x2b21 bytes) 2024-04-23T23:50:09.060 INFO:teuthology.orchestra.run.smithi092.stdout:1294 writesame 0xcd28e9b thru 0xcd3869b data_size 0x1(0xf801 bytes) 2024-04-23T23:50:09.101 INFO:teuthology.orchestra.run.smithi092.stdout:1295 read 0x273194 thru 0x27bf2b (0x8d98 bytes) 2024-04-23T23:50:09.102 INFO:teuthology.orchestra.run.smithi092.stdout:1296 punch from 0xec9406 to 0xed8306, (0xef00 bytes) 2024-04-23T23:50:09.114 INFO:teuthology.orchestra.run.smithi092.stdout:1297 trunc from 0xce5c09c to 0x29d1873 2024-04-23T23:50:09.126 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:09.140+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:09.139 INFO:teuthology.orchestra.run.smithi092.stdout:1298 write 0x7477fbd thru 0x7485d39 (0xdd7d bytes) 2024-04-23T23:50:09.143 INFO:teuthology.orchestra.run.smithi092.stdout:1299 write 0x2a88494 thru 0x2a8c5a2 (0x410f bytes) 2024-04-23T23:50:09.144 INFO:teuthology.orchestra.run.smithi092.stdout:1300 read 0x7170be9 thru 0x717e6e5 (0xdafd bytes) 2024-04-23T23:50:09.145 INFO:teuthology.orchestra.run.smithi092.stdout:1301 punch from 0x1a42a18 to 0x1a43115, (0x6fd bytes) 2024-04-23T23:50:09.145 INFO:teuthology.orchestra.run.smithi092.stdout:1306 read 0x34302b8 thru 0x343512a (0x4e73 bytes) 2024-04-23T23:50:09.148 INFO:teuthology.orchestra.run.smithi092.stdout:1307 trunc from 0x7485d3a to 0x9c4eace 2024-04-23T23:50:09.154 INFO:teuthology.orchestra.run.smithi092.stdout:1308 read 0x4280d81 thru 0x428a36f (0x95ef bytes) 2024-04-23T23:50:09.154 INFO:teuthology.orchestra.run.smithi092.stdout:1310 read 0x991e386 thru 0x991f34c (0xfc7 bytes) 2024-04-23T23:50:09.154 INFO:teuthology.orchestra.run.smithi092.stdout:1311 read 0x42cb975 thru 0x42d1c91 (0x631d bytes) 2024-04-23T23:50:09.160 INFO:teuthology.orchestra.run.smithi092.stdout:1312 trunc from 0x9c4eace to 0x4936511 2024-04-23T23:50:09.170 INFO:teuthology.orchestra.run.smithi092.stdout:1314 read 0x3657b38 thru 0x3660b6d (0x9036 bytes) 2024-04-23T23:50:09.170 INFO:teuthology.orchestra.run.smithi092.stdout:1315 read 0x47a5a77 thru 0x47aa74f (0x4cd9 bytes) 2024-04-23T23:50:09.170 INFO:teuthology.orchestra.run.smithi092.stdout:1316 trunc from 0x4936511 to 0x44a09d1 2024-04-23T23:50:09.191 INFO:teuthology.orchestra.run.smithi092.stdout:1317 writesame 0xd44ad05 thru 0xd454588 data_size 0x4(0x9884 bytes) 2024-04-23T23:50:09.191 INFO:teuthology.orchestra.run.smithi092.stdout:1318 trunc from 0xd454589 to 0xdeb9de1 2024-04-23T23:50:09.196 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.196 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.197 INFO:teuthology.orchestra.run.smithi092.stdout:1320 write 0x9bda8ff thru 0x9be9347 (0xea49 bytes) 2024-04-23T23:50:09.201 INFO:teuthology.orchestra.run.smithi092.stdout:1322 read 0xdeaffd4 thru 0xdeb4aad (0x4ada bytes) 2024-04-23T23:50:09.201 INFO:teuthology.orchestra.run.smithi092.stdout:1323 writesame 0xd141749 thru 0xd145a69 data_size 0x7(0x4321 bytes) 2024-04-23T23:50:09.204 INFO:teuthology.orchestra.run.smithi092.stdout:1325 write 0xa8a3a85 thru 0xa8af4e2 (0xba5e bytes) 2024-04-23T23:50:09.205 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:09.218+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:09.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:09.206 INFO:teuthology.orchestra.run.smithi092.stdout:1326 write 0x22ef314 thru 0x22f104e (0x1d3b bytes) 2024-04-23T23:50:09.215 INFO:teuthology.orchestra.run.smithi092.stdout:1327 trunc from 0xdeb9de1 to 0x7840d82 2024-04-23T23:50:09.224 INFO:teuthology.orchestra.run.smithi092.stdout:1328 read 0xf5b785 thru 0xf5c532 (0xdae bytes) 2024-04-23T23:50:09.225 INFO:teuthology.orchestra.run.smithi092.stdout:1330 trunc from 0x7840d82 to 0x6d26622 2024-04-23T23:50:09.231 INFO:teuthology.orchestra.run.smithi092.stdout:1331 trunc from 0x6d26622 to 0x789075b 2024-04-23T23:50:09.237 INFO:teuthology.orchestra.run.smithi092.stdout:1332 write 0x504ef72 thru 0x5055c13 (0x6ca2 bytes) 2024-04-23T23:50:09.240 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.240 INFO:teuthology.orchestra.run.smithi092.stdout:1336 punch from 0x3c675f7 to 0x3c7662a, (0xf033 bytes) 2024-04-23T23:50:09.240 INFO:teuthology.orchestra.run.smithi092.stdout:1337 write 0x6bbb468 thru 0x6bc8e54 (0xd9ed bytes) 2024-04-23T23:50:09.247 INFO:teuthology.orchestra.run.smithi092.stdout:1339 write 0x63b89eb thru 0x63c4a0d (0xc023 bytes) 2024-04-23T23:50:09.250 INFO:teuthology.orchestra.run.smithi092.stdout:1342 write 0x6bbcf5d thru 0x6bc4296 (0x733a bytes) 2024-04-23T23:50:09.251 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.252 INFO:teuthology.orchestra.run.smithi092.stdout:1346 read 0xe888a thru 0xf67c2 (0xdf39 bytes) 2024-04-23T23:50:09.261 INFO:teuthology.orchestra.run.smithi092.stdout:1347 write 0x99532d6 thru 0x995649b (0x31c6 bytes) 2024-04-23T23:50:09.264 INFO:teuthology.orchestra.run.smithi092.stdout:1348 writesame 0x2851861 thru 0x285e268 data_size 0x3c(0xca08 bytes) 2024-04-23T23:50:09.266 INFO:teuthology.orchestra.run.smithi092.stdout:1349 write 0x95aa451 thru 0x95b1f94 (0x7b44 bytes) 2024-04-23T23:50:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:1351 write 0x5ee6865 thru 0x5ef4080 (0xd81c bytes) 2024-04-23T23:50:09.271 INFO:teuthology.orchestra.run.smithi092.stdout:1352 read 0x8360b3d thru 0x836364b (0x2b0f bytes) 2024-04-23T23:50:09.282 INFO:teuthology.orchestra.run.smithi092.stdout:1353 write 0xd080936 thru 0xd084cc0 (0x438b bytes) 2024-04-23T23:50:09.294 INFO:teuthology.orchestra.run.smithi092.stdout:1355 trunc from 0xd084cc1 to 0x4829f52 2024-04-23T23:50:09.306 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:09.319+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:09.309 INFO:teuthology.orchestra.run.smithi092.stdout:1356 write 0x904483 thru 0x90eb75 (0xa6f3 bytes) 2024-04-23T23:50:09.310 INFO:teuthology.orchestra.run.smithi092.stdout:1357 punch from 0x37b53ab to 0x37c02ad, (0xaf02 bytes) 2024-04-23T23:50:09.323 INFO:teuthology.orchestra.run.smithi092.stdout:1358 write 0xa64608c thru 0xa64c694 (0x6609 bytes) 2024-04-23T23:50:09.326 INFO:teuthology.orchestra.run.smithi092.stdout:1361 punch from 0x735a909 to 0x735ba33, (0x112a bytes) 2024-04-23T23:50:09.326 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.326 INFO:teuthology.orchestra.run.smithi092.stdout:1363 read 0x998b945 thru 0x9996b21 (0xb1dd bytes) 2024-04-23T23:50:09.327 INFO:teuthology.orchestra.run.smithi092.stdout:1364 writesame 0xa937dd thru 0xa93941 data_size 0x7(0x165 bytes) 2024-04-23T23:50:09.328 INFO:teuthology.orchestra.run.smithi092.stdout:1365 write 0x447a1a8 thru 0x4480f5a (0x6db3 bytes) 2024-04-23T23:50:09.337 INFO:teuthology.orchestra.run.smithi092.stdout:1367 write 0xaa810a6 thru 0xaa8e06c (0xcfc7 bytes) 2024-04-23T23:50:09.340 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.351 INFO:teuthology.orchestra.run.smithi092.stdout:1369 trunc from 0xaa8e06d to 0x1708f25 2024-04-23T23:50:09.358 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.358 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.364 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.364 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.365 INFO:teuthology.orchestra.run.smithi092.stdout:1373 trunc from 0x1708f25 to 0x1bdfe3 2024-04-23T23:50:09.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:09.383 INFO:teuthology.orchestra.run.smithi092.stdout:1374 writesame 0x235325f thru 0x235c3e3 data_size 0x1(0x9185 bytes) 2024-04-23T23:50:09.384 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:1377 writesame 0x84ff3cc thru 0x850bdbb data_size 0x48(0xc9f0 bytes) 2024-04-23T23:50:09.398 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.398 INFO:teuthology.orchestra.run.smithi092.stdout:1379 punch from 0x18e44f6 to 0x18f2a8c, (0xe596 bytes) 2024-04-23T23:50:09.398 INFO:teuthology.orchestra.run.smithi092.stdout:1380 read 0x1b62b47 thru 0x1b65646 (0x2b00 bytes) 2024-04-23T23:50:09.399 INFO:teuthology.orchestra.run.smithi092.stdout:1381 trunc from 0x850bdbc to 0x95e4a7c 2024-04-23T23:50:09.405 INFO:teuthology.orchestra.run.smithi092.stdout:1382 read 0x31f5b7f thru 0x31f62db (0x75d bytes) 2024-04-23T23:50:09.405 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.405 INFO:teuthology.orchestra.run.smithi092.stdout:1384 write 0x1da292c thru 0x1da7e31 (0x5506 bytes) 2024-04-23T23:50:09.406 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:09.420+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:09.406 INFO:teuthology.orchestra.run.smithi092.stdout:1385 read 0xd55631 thru 0xd589be (0x338e bytes) 2024-04-23T23:50:09.407 INFO:teuthology.orchestra.run.smithi092.stdout:1387 punch from 0x26a9c43 to 0x26b8691, (0xea4e bytes) 2024-04-23T23:50:09.407 INFO:teuthology.orchestra.run.smithi092.stdout:1388 write 0x7ecf915 thru 0x7ed0e19 (0x1505 bytes) 2024-04-23T23:50:09.420 INFO:teuthology.orchestra.run.smithi092.stdout:1390 writesame 0xcc80c47 thru 0xcc81840 data_size 0x15(0xbfa bytes) 2024-04-23T23:50:09.420 INFO:teuthology.orchestra.run.smithi092.stdout:1391 write 0x9d2fd98 thru 0x9d33da7 (0x4010 bytes) 2024-04-23T23:50:09.426 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.426 INFO:teuthology.orchestra.run.smithi092.stdout:1393 write 0x3e6c30c thru 0x3e78c54 (0xc949 bytes) 2024-04-23T23:50:09.426 INFO:teuthology.orchestra.run.smithi092.stdout:1394 punch from 0x53b4c64 to 0x53b85d5, (0x3971 bytes) 2024-04-23T23:50:09.426 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.427 INFO:teuthology.orchestra.run.smithi092.stdout:1397 punch from 0x87acd4c to 0x87b7681, (0xa935 bytes) 2024-04-23T23:50:09.427 INFO:teuthology.orchestra.run.smithi092.stdout:1400 flatten 2024-04-23T23:50:09.433 INFO:teuthology.orchestra.run.smithi092.stdout:1401 read 0x4babe98 thru 0x4bae984 (0x2aed bytes) 2024-04-23T23:50:09.438 INFO:teuthology.orchestra.run.smithi092.stdout:1403 write 0xcf19e1a thru 0xcf1c52a (0x2711 bytes) 2024-04-23T23:50:09.441 INFO:teuthology.orchestra.run.smithi092.stdout:1404 writesame 0x8cad160 thru 0x8cb8f0f data_size 0x28(0xbdb0 bytes) 2024-04-23T23:50:09.443 INFO:teuthology.orchestra.run.smithi092.stdout:1405 read 0x9881297 thru 0x988da4c (0xc7b6 bytes) 2024-04-23T23:50:09.443 INFO:teuthology.orchestra.run.smithi092.stdout:1406 punch from 0x3c0244d to 0x3c0df4e, (0xbb01 bytes) 2024-04-23T23:50:09.443 INFO:teuthology.orchestra.run.smithi092.stdout:1407 write 0x787a650 thru 0x7884ca5 (0xa656 bytes) 2024-04-23T23:50:09.446 INFO:teuthology.orchestra.run.smithi092.stdout:1408 punch from 0xc92c6ae to 0xc938bee, (0xc540 bytes) 2024-04-23T23:50:09.446 INFO:teuthology.orchestra.run.smithi092.stdout:1411 write 0xbce2119 thru 0xbce77b3 (0x569b bytes) 2024-04-23T23:50:09.463 INFO:teuthology.orchestra.run.smithi092.stdout:1413 trunc from 0xcf1c52b to 0x14a0c80 2024-04-23T23:50:09.479 INFO:teuthology.orchestra.run.smithi092.stdout:1414 writesame 0x1eba47c thru 0x1ec653a data_size 0x35(0xc0bf bytes) 2024-04-23T23:50:09.480 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.480 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.480 INFO:teuthology.orchestra.run.smithi092.stdout:1418 read 0x1780083 thru 0x17878c3 (0x7841 bytes) 2024-04-23T23:50:09.490 INFO:teuthology.orchestra.run.smithi092.stdout:1419 write 0x488969c thru 0x488ab51 (0x14b6 bytes) 2024-04-23T23:50:09.499 INFO:teuthology.orchestra.run.smithi092.stdout:1420 trunc from 0x488ab52 to 0x9dbe954 2024-04-23T23:50:09.507 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:09.521+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:09.511 INFO:teuthology.orchestra.run.smithi092.stdout:1421 write 0xa2861f3 thru 0xa28f6dd (0x94eb bytes) 2024-04-23T23:50:09.514 INFO:teuthology.orchestra.run.smithi092.stdout:1422 punch from 0xa171a75 to 0xa17627e, (0x4809 bytes) 2024-04-23T23:50:09.514 INFO:teuthology.orchestra.run.smithi092.stdout:1423 write 0x9a9e7fc thru 0x9aa6664 (0x7e69 bytes) 2024-04-23T23:50:09.517 INFO:teuthology.orchestra.run.smithi092.stdout:1424 punch from 0x7331067 to 0x7332ecf, (0x1e68 bytes) 2024-04-23T23:50:09.519 INFO:teuthology.orchestra.run.smithi092.stdout:1425 trunc from 0xa28f6de to 0x802d342 2024-04-23T23:50:09.526 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.526 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.530 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.530 INFO:teuthology.orchestra.run.smithi092.stdout:1428 punch from 0x171d558 to 0x1721487, (0x3f2f bytes) 2024-04-23T23:50:09.530 INFO:teuthology.orchestra.run.smithi092.stdout:1429 write 0x5a61bee thru 0x5a634c0 (0x18d3 bytes) 2024-04-23T23:50:09.532 INFO:teuthology.orchestra.run.smithi092.stdout:1430 read 0x77c5241 thru 0x77cb4ea (0x62aa bytes) 2024-04-23T23:50:09.536 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:09.541 INFO:teuthology.orchestra.run.smithi092.stdout:1431 write 0x97d41a5 thru 0x97e2dd4 (0xec30 bytes) 2024-04-23T23:50:09.547 INFO:teuthology.orchestra.run.smithi092.stdout:1432 punch from 0x2446b73 to 0x244c65c, (0x5ae9 bytes) 2024-04-23T23:50:09.547 INFO:teuthology.orchestra.run.smithi092.stdout:1433 read 0x7fc4791 thru 0x7fd322e (0xea9e bytes) 2024-04-23T23:50:09.547 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.547 INFO:teuthology.orchestra.run.smithi092.stdout:1437 read 0x547e3d4 thru 0x548ba4d (0xd67a bytes) 2024-04-23T23:50:09.548 INFO:teuthology.orchestra.run.smithi092.stdout:1439 read 0x951068b thru 0x951cab6 (0xc42c bytes) 2024-04-23T23:50:09.548 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:09.548 INFO:teuthology.orchestra.run.smithi092.stdout:1441 write 0x185093a thru 0x185486e (0x3f35 bytes) 2024-04-23T23:50:09.559 INFO:teuthology.orchestra.run.smithi092.stdout:1444 write 0xcb3d818 thru 0xcb49fa2 (0xc78b bytes) 2024-04-23T23:50:09.570 INFO:teuthology.orchestra.run.smithi092.stdout:1445 write 0xe2b244d thru 0xe2b8e63 (0x6a17 bytes) 2024-04-23T23:50:09.574 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-23T23:50:09.574 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:09.574 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-23T23:50:09.575 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-23T23:50:09.575 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:50:09.581 INFO:teuthology.orchestra.run.smithi092.stdout:1446 trunc from 0xe2b8e64 to 0x7ba1964 2024-04-23T23:50:09.603 INFO:tasks.ceph.osd.3:Stopped 2024-04-23T23:50:09.604 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-04-23T23:50:09.604 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:50:09.604 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-04-23T23:50:09.608 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:09.622+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:09.643 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.644 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:09.658 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-23T23:50:09.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-04-23T23:50:09.737 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:09.751+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:09.799 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.799 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:09.809 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:09.823+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:09.910 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:09.924+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:09.952 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:09.952 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:09.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:09.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:09.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:50:10.011 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:10.024+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:10.066 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:10.066 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:10.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:10.100 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:10.111 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:10.125+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:10.136 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:10.156 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:10.176 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:10.195 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:10.212 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:10.212 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:10.226+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:10.215 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:10.215 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.034482523000001, 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.685745855, 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6857294019999998, 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6857026610000001, 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:10.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.243 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6856838659999998, 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:10.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6856662820000001, 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:10.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.252 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6739707780000002, 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6737950910000001, 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:10.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:10.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:10.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.673643572, 2024-04-23T23:50:10.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:10.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.265 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6733609459999998, 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:10.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:10.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6732150809999999, 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.270 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.274 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6722653969999999, 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:10.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6722425030000001, 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:10.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.283 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6562271280000003, 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:10.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:10.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:10.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.287 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6562174939999998, 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:10.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.641185133, 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.292 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:10.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.641163809, 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.6411409839999997, 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:10.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:10.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:10.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.641118305, 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:10.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:10.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:10.309 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.638816582, 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:10.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:10.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:10.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:10.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:10.315 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:10.316 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:10.330+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:10.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:10.418 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:10.432+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:10.419 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:10.419 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:10.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:10.518 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:10.532+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:10.534 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:10.535 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:10.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:10.619 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:10.633+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:10.650 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10283 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7225344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3904029109999998, 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035444584000000001, 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10283 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273274+0000", 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273279+0000", 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5820000000000001e-06 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273284+0000", 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308551+0000", 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035267169000000001 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308626+0000", 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4328999999999999e-05 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308716+0000", 2024-04-23T23:50:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0647000000000001e-05 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10284 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7389184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3903848760000002, 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035618043000000002, 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10284 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273292+0000", 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9360000000000002e-06 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273298+0000", 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6860000000000003e-06 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273303+0000", 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1830000000000002e-06 2024-04-23T23:50:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308748+0000", 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035444749999999997 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308824+0000", 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6705999999999994e-05 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308908+0000", 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10285 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3903562580000002, 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035744949999999998, 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10285 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273319+0000", 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6000000000000004e-07 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273322+0000", 2024-04-23T23:50:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.236e-06 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273329+0000", 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1350000000000003e-06 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273335+0000", 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5149999999999997e-06 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308939+0000", 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035603856000000003 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309006+0000", 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6934999999999995e-05 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309063+0000", 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7713000000000001e-05 2024-04-23T23:50:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10286 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3903302389999999, 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035978035999999998, 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10286 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273345+0000", 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273347+0000", 2024-04-23T23:50:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.125e-06 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273363+0000", 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6353000000000001e-05 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273370+0000", 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309119+0000", 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035748874 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309212+0000", 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2523999999999997e-05 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309322+0000", 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110597 2024-04-23T23:50:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10287 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8142848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3902959350000001, 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036157685000000002, 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10287 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273382+0000", 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4959999999999999e-06 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273389+0000", 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9160000000000004e-06 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273394+0000", 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1710000000000001e-06 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309383+0000", 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035989128000000002 2024-04-23T23:50:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309448+0000", 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5419999999999994e-05 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309536+0000", 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8034000000000005e-05 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10288 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8306688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3902636390000001, 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036258020000000002, 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10288 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273414+0000", 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7259999999999998e-06 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273422+0000", 2024-04-23T23:50:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5390000000000002e-06 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273427+0000", 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309563+0000", 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036135689999999998 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309618+0000", 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4994000000000002e-05 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309669+0000", 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1147999999999999e-05 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10289 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8470528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3902304710000002, 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036365322999999998, 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10289 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273445+0000", 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273447+0000", 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.587e-06 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273456+0000", 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6770000000000001e-06 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273462+0000", 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8810000000000001e-06 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309690+0000", 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036228306000000002 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309753+0000", 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2772000000000001e-05 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309809+0000", 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6560000000000001e-05 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10290 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3902051530000001, 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036521275999999998, 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10290 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273470+0000", 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-07 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273472+0000", 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2330000000000001e-06 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273479+0000", 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4080000000000003e-06 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273484+0000", 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9819999999999999e-06 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309838+0000", 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036354356999999997 2024-04-23T23:50:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309923+0000", 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5130000000000007e-05 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309991+0000", 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7630000000000001e-05 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10291 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9060352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3901820730000001, 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036628916999999997, 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10291 2024-04-23T23:50:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0900000000000002e-07 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273495+0000", 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3190000000000002e-06 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273502+0000", 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4609999999999996e-06 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273507+0000", 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9490000000000004e-06 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310011+0000", 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036504395000000002 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310057+0000", 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6008999999999999e-05 2024-04-23T23:50:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310121+0000", 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4275000000000004e-05 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10292 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9224192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3901304319999999, 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036703116000000001, 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10292 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273545+0000", 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273548+0000", 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273557+0000", 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2979999999999997e-06 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273563+0000", 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7810000000000002e-06 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310148+0000", 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036585212999999998 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310198+0000", 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0686999999999998e-05 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310247+0000", 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8801000000000003e-05 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10293 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9388032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.390023641, 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036741629999999997, 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10293 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273652+0000", 2024-04-23T23:50:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273665+0000", 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.34e-05 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273673+0000", 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3070000000000003e-06 2024-04-23T23:50:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273679+0000", 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8610000000000003e-06 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310264+0000", 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036584942000000002 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310304+0000", 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9932999999999999e-05 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310393+0000", 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8640999999999994e-05 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10294 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9551872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3899869730000001, 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036864814000000003, 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10294 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273690+0000", 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8139999999999999e-06 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273696+0000", 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7760000000000003e-06 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273701+0000", 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0159999999999999e-06 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310423+0000", 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036722312999999999 2024-04-23T23:50:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310489+0000", 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5741000000000006e-05 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310552+0000", 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3674000000000001e-05 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10295 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9977856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3899667230000001, 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036991522999999998, 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10295 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.77e-07 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273710+0000", 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0829999999999998e-06 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273716+0000", 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9410000000000004e-06 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273721+0000", 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.955e-06 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310584+0000", 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036862930000000002 2024-04-23T23:50:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310644+0000", 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0037000000000001e-05 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310699+0000", 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.52e-05 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10296 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10141696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3899448620000001, 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037192338999999998, 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10296 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273732+0000", 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2024-04-23T23:50:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273739+0000", 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5599999999999999e-06 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273744+0000", 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0980000000000001e-06 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310736+0000", 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036992522999999999 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310811+0000", 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4184999999999995e-05 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310922+0000", 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111532 2024-04-23T23:50:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10297 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10305536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3899137339999998, 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037380582000000002, 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10297 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273764+0000", 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273790+0000", 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273797+0000", 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0129999999999996e-06 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310950+0000", 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037152875000000002 2024-04-23T23:50:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311040+0000", 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9816999999999997e-05 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311141+0000", 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010139700000000001 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10298 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10469376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3898684929999998, 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037503460000000002, 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10298 2024-04-23T23:50:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273807+0000", 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8800000000000003e-07 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273809+0000", 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273816+0000", 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5830000000000003e-06 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273821+0000", 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.848e-06 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311188+0000", 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037366982 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311234+0000", 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6269e-05 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311310+0000", 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780999999999998e-05 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10299 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10895360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3898473930000002, 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037660666000000002, 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10299 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273828+0000", 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273835+0000", 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273844+0000", 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1360000000000003e-06 2024-04-23T23:50:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273850+0000", 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3360000000000002e-06 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311345+0000", 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037495406000000002 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311415+0000", 2024-04-23T23:50:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9605999999999997e-05 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311488+0000", 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3161e-05 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10300 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11059200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3897838240000002, 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037811175000000002, 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10300 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2200000000000004e-07 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273894+0000", 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9550000000000001e-06 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273903+0000", 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3299999999999999e-06 2024-04-23T23:50:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273910+0000", 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8689999999999998e-06 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311574+0000", 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037664133000000002 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311616+0000", 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2481000000000002e-05 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311702+0000", 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5785000000000006e-05 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10301 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11223040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3897566989999999, 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037936401000000002, 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10301 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.44e-07 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273921+0000", 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273927+0000", 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5270000000000004e-06 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273932+0000", 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.215e-06 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311737+0000", 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037804799 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311801+0000", 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3652999999999995e-05 2024-04-23T23:50:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311854+0000", 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3542999999999999e-05 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10302 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11386880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3897335439999998, 2024-04-23T23:50:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038084931000000002, 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10302 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273944+0000", 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273950+0000", 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6540000000000002e-06 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273956+0000", 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311874+0000", 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037918232000000003 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311951+0000", 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6701999999999998e-05 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.312026+0000", 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5469000000000005e-05 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:10.753 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:10.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:10.755 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:10.768+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.c 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.7 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.2 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.8 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.1e 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.b 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.5 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.18 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.15 2024-04-23T23:50:10.859 INFO:teuthology.orchestra.run.smithi092.stdout:3.19 2024-04-23T23:50:10.860 INFO:teuthology.orchestra.run.smithi092.stdout:2.1 2024-04-23T23:50:10.860 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:10.872+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:10.863 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:10.864 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:10.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:10.959 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:10.973+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:10.965 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:50:10.981 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:10.981 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:10.984 INFO:teuthology.orchestra.run.smithi092.stdout:1449 read 0x5e725bb thru 0x5e792c0 (0x6d06 bytes) 2024-04-23T23:50:10.984 INFO:teuthology.orchestra.run.smithi092.stdout:1450 punch from 0x654dabd to 0x655abfb, (0xd13e bytes) 2024-04-23T23:50:10.990 INFO:teuthology.orchestra.run.smithi092.stdout:1451 trunc from 0x7ba1964 to 0xd7e15e6 2024-04-23T23:50:10.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:10.996 INFO:teuthology.orchestra.run.smithi092.stdout:1452 write 0x79eef1e thru 0x79f0e47 (0x1f2a bytes) 2024-04-23T23:50:10.999 INFO:teuthology.orchestra.run.smithi092.stdout:1453 writesame 0xadcc1ec thru 0xadd77d6 data_size 0x1(0xb5eb bytes) 2024-04-23T23:50:11.025 INFO:teuthology.orchestra.run.smithi092.stdout:1454 write 0xea87be thru 0xeb0fef (0x8832 bytes) 2024-04-23T23:50:11.026 INFO:teuthology.orchestra.run.smithi092.stdout:1455 write 0x440a69c thru 0x440b651 (0xfb6 bytes) 2024-04-23T23:50:11.027 INFO:teuthology.orchestra.run.smithi092.stdout:1456 writesame 0xa5ff1e6 thru 0xa60e54d data_size 0x4(0xf368 bytes) 2024-04-23T23:50:11.040 INFO:teuthology.orchestra.run.smithi092.stdout:1457 write 0xd36c6f thru 0xd3a142 (0x34d4 bytes) 2024-04-23T23:50:11.041 INFO:teuthology.orchestra.run.smithi092.stdout:1458 writesame 0x2c74d7d thru 0x2c7d128 data_size 0x6a(0x83ac bytes) 2024-04-23T23:50:11.054 INFO:teuthology.orchestra.run.smithi092.stdout:1459 trunc from 0xd7e15e6 to 0x4715131 2024-04-23T23:50:11.060 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:11.074+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:11.081 INFO:teuthology.orchestra.run.smithi092.stdout:1460 writesame 0xb987707 thru 0xb9941f3 data_size 0x1(0xcaed bytes) 2024-04-23T23:50:11.081 INFO:teuthology.orchestra.run.smithi092.stdout:1461 writesame 0x5150389 thru 0x5157ac5 data_size 0x4b(0x773d bytes) 2024-04-23T23:50:11.085 INFO:teuthology.orchestra.run.smithi092.stdout:1462 read 0x8098f5d thru 0x809e1e7 (0x528b bytes) 2024-04-23T23:50:11.085 INFO:teuthology.orchestra.run.smithi092.stdout:1463 read 0x6900945 thru 0x690dc3f (0xd2fb bytes) 2024-04-23T23:50:11.085 INFO:teuthology.orchestra.run.smithi092.stdout:1464 read 0xab76b07 thru 0xab77c3c (0x1136 bytes) 2024-04-23T23:50:11.085 INFO:teuthology.orchestra.run.smithi092.stdout:1465 write 0xd8a38c thru 0xd98538 (0xe1ad bytes) 2024-04-23T23:50:11.088 INFO:teuthology.orchestra.run.smithi092.stdout:1466 trunc from 0xb9941f4 to 0xddbbbf1 2024-04-23T23:50:11.094 INFO:teuthology.orchestra.run.smithi092.stdout:1467 writesame 0xc0dcce0 thru 0xc0de153 data_size 0x2c(0x1474 bytes) 2024-04-23T23:50:11.096 INFO:teuthology.orchestra.run.smithi092.stdout:1468 write 0x765f2b6 thru 0x7664e4d (0x5b98 bytes) 2024-04-23T23:50:11.100 INFO:teuthology.orchestra.run.smithi092.stdout:1469 write 0xca0cb8a thru 0xca18e31 (0xc2a8 bytes) 2024-04-23T23:50:11.114 INFO:teuthology.orchestra.run.smithi092.stdout:1471 trunc from 0xddbbbf1 to 0x443076d 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:3037 3.4 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head [stat,truncate 5242880] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6704647169999998, 2024-04-23T23:50:11.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089969830000000001, 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 3037 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456900+0000", 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0860000000000001e-06 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456911+0000", 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1467e-05 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456922+0000", 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380999999999999e-05 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456929+0000", 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8000000000000001e-06 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459092+0000", 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002163697 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459193+0000", 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100709 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459396+0000", 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020303399999999999 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.462879+0000", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034833899999999998 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465827+0000", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029476889999999999 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465863+0000", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5503000000000002e-05 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465896+0000", 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3226999999999999e-05 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6867 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8192843080000003, 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077717510000000004, 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6867 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308080+0000", 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308082+0000", 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308088+0000", 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308092+0000", 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6140000000000002e-06 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315699+0000", 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076064569999999996 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315761+0000", 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2194999999999997e-05 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315851+0000", 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0109000000000001e-05 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.125 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6868 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8192652279999999, 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079224340000000008, 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6868 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308099+0000", 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4099999999999999e-07 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308101+0000", 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9080000000000002e-06 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308106+0000", 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6869999999999999e-06 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308111+0000", 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315874+0000", 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077625799999999998 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315936+0000", 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2442999999999998e-05 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316021+0000", 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4592999999999995e-05 2024-04-23T23:50:11.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6871 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.819246562, 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080708470000000008, 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6871 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308119+0000", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8509999999999999e-06 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308128+0000", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2509999999999994e-06 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308133+0000", 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316044+0000", 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079107259999999999 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316108+0000", 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3937999999999997e-05 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316188+0000", 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0208000000000002e-05 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6872 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8192249340000002, 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082303659999999994, 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6872 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308142+0000", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.79e-06 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308147+0000", 2024-04-23T23:50:11.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3639999999999998e-06 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308151+0000", 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.34e-06 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316212+0000", 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080606099999999993 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316280+0000", 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7692000000000002e-05 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316369+0000", 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9148999999999996e-05 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6875 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8192062440000001, 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083900060000000002, 2024-04-23T23:50:11.138 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6875 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308158+0000", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308160+0000", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9290000000000001e-06 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308166+0000", 2024-04-23T23:50:11.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6510000000000004e-06 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308171+0000", 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2640000000000002e-06 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316398+0000", 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082269720000000008 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316466+0000", 2024-04-23T23:50:11.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8361999999999994e-05 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316547+0000", 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1311999999999997e-05 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6876 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8191864439999996, 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085708390000000002, 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6876 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:11.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308178+0000", 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308179+0000", 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.844e-06 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308185+0000", 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.198e-06 2024-04-23T23:50:11.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308189+0000", 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1699999999999999e-06 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316574+0000", 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083852639999999999 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316652+0000", 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8078999999999996e-05 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316748+0000", 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5784000000000003e-05 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6878 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8191666059999996, 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087314869999999996, 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6878 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308199+0000", 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0559999999999999e-06 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308205+0000", 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8329999999999999e-06 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308210+0000", 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4519999999999999e-06 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316773+0000", 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085637890000000005 2024-04-23T23:50:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316846+0000", 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2389000000000001e-05 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316928+0000", 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2602000000000006e-05 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6880 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8191482150000002, 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089037119999999994, 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6880 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308216+0000", 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-04-23T23:50:11.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308224+0000", 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4409999999999993e-06 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308230+0000", 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8340000000000004e-06 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308234+0000", 2024-04-23T23:50:11.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4109999999999998e-06 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316953+0000", 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087181989999999994 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317029+0000", 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6186999999999994e-05 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317119+0000", 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0286999999999998e-05 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6883 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.819112466, 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090432810000000002, 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6883 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308252+0000", 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4199999999999995e-07 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308254+0000", 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4940000000000002e-06 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308262+0000", 2024-04-23T23:50:11.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3780000000000003e-06 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308267+0000", 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8540000000000002e-06 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317145+0000", 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088778469999999995 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317212+0000", 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6686000000000005e-05 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317294+0000", 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2379999999999994e-05 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6884 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8190897870000002, 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092106370000000007, 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6884 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2500000000000001e-07 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308276+0000", 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.978e-06 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308282+0000", 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308286+0000", 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3649999999999997e-06 2024-04-23T23:50:11.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317320+0000", 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009034172 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317394+0000", 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3825000000000006e-05 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317484+0000", 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0340000000000006e-05 2024-04-23T23:50:11.163 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6886 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8190719670000002, 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093837590000000002, 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6886 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308294+0000", 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.86e-06 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308300+0000", 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308305+0000", 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3249999999999998e-06 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317511+0000", 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092054749999999994 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317587+0000", 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6179000000000004e-05 2024-04-23T23:50:11.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317675+0000", 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8595000000000001e-05 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6890 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8190528649999997, 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095370109999999998, 2024-04-23T23:50:11.168 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6890 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308313+0000", 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8989999999999999e-06 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308320+0000", 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.827e-06 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308324+0000", 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2830000000000003e-06 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317700+0000", 2024-04-23T23:50:11.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093762359999999996 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317767+0000", 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6550000000000005e-05 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317848+0000", 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0844e-05 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6892 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8190339919999996, 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097323389999999996, 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6892 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-04-23T23:50:11.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308332+0000", 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.883e-06 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308338+0000", 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9680000000000003e-06 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308342+0000", 2024-04-23T23:50:11.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.143e-06 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317873+0000", 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095314449999999995 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317937+0000", 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3165999999999998e-05 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318062+0000", 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012526900000000001 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6893 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8190164739999997, 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099354839999999996, 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6893 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308348+0000", 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0399999999999996e-07 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308349+0000", 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7540000000000001e-06 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308355+0000", 2024-04-23T23:50:11.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3419999999999998e-06 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308359+0000", 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4959999999999998e-06 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318107+0000", 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097478240000000004 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318189+0000", 2024-04-23T23:50:11.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1762999999999996e-05 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318283+0000", 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3801000000000005e-05 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6894 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8189990759999999, 2024-04-23T23:50:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126606, 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6894 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308365+0000", 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308367+0000", 2024-04-23T23:50:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8920000000000001e-06 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308373+0000", 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8939999999999998e-06 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308377+0000", 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2529999999999998e-06 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318324+0000", 2024-04-23T23:50:11.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099474309999999996 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318393+0000", 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8727999999999997e-05 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318491+0000", 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8036999999999997e-05 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6897 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8189814560000004, 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010295080999999999, 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6897 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.186 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308383+0000", 2024-04-23T23:50:11.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308384+0000", 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.832e-06 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308396+0000", 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1208e-05 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308400+0000", 2024-04-23T23:50:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.639e-06 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318519+0000", 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010119025 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318592+0000", 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3124e-05 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318677+0000", 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4792999999999999e-05 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6898 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8189573189999999, 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010449478999999999, 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6898 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308407+0000", 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308409+0000", 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308414+0000", 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5230000000000003e-06 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308418+0000", 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2039999999999999e-06 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318706+0000", 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01028717 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318774+0000", 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7983000000000003e-05 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318856+0000", 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2047999999999998e-05 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6900 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:11.194 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8189394199999995, 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010615769000000001, 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6900 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308427+0000", 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1789999999999998e-06 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308432+0000", 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5990000000000001e-06 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308440+0000", 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318884+0000", 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010443913000000001 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318953+0000", 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8650000000000002e-05 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319040+0000", 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6908000000000002e-05 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6902 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.818916432, 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010766184, 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6902 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.199 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:11.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308455+0000", 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0239999999999997e-06 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308462+0000", 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-06 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308471+0000", 2024-04-23T23:50:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4509999999999998e-06 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319063+0000", 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010591685999999999 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319132+0000", 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8628999999999996e-05 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319213+0000", 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1453999999999996e-05 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.203 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:11.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1472 write 0x669d7b4 thru 0x66a4d9a (0x75e7 bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1475 punch from 0x1b219ae to 0x1b2f80a, (0xde5c bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1476 write 0x3678e68 thru 0x367eff6 (0x618f bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1477 punch from 0x37abb4f to 0x37bb094, (0xf545 bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1478 writesame 0x454e303 thru 0x4556f5e data_size 0x1a(0x8c5c bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1480 write 0x9c65a7f thru 0x9c6ae29 (0x53ab bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1481 read 0x82609e3 thru 0x826afbe (0xa5dc bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1482 read 0x3187d48 thru 0x3196350 (0xe609 bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1483 read 0x7645f4a thru 0x764e5ff (0x86b6 bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1484 read 0x5ba0c88 thru 0x5bb0532 (0xf8ab bytes) 2024-04-23T23:50:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:1485 trunc from 0x9c6ae2a to 0x205c63f 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1487 punch from 0x663e3d to 0x673e36, (0xfff9 bytes) 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1490 read 0x1b32d3 thru 0x1b6513 (0x3241 bytes) 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1491 punch from 0x58df99 to 0x599a31, (0xba98 bytes) 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1492 read 0x2cd8d4 thru 0x2dd4f0 (0xfc1d bytes) 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1493 writesame 0xddf6152 thru 0xddff0b4 data_size 0xb(0x8f63 bytes) 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1495 punch from 0x2f79b4b to 0x2f7abb6, (0x106b bytes) 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1496 trunc from 0xddff0b5 to 0x8ea556a 2024-04-23T23:50:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:1497 read 0x2e94419 thru 0x2ea405e (0xfc46 bytes) 2024-04-23T23:50:11.208 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:11.218+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:1499 writesame 0xbffd0f9 thru 0xc00bca0 data_size 0x4(0xeba8 bytes) 2024-04-23T23:50:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:1501 punch from 0xa539d8 to 0xa60752, (0xcd7a bytes) 2024-04-23T23:50:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:1502 read 0xa6f793d thru 0xa6f8ce9 (0x13ad bytes) 2024-04-23T23:50:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:1503 writesame 0x8747b5b thru 0x874c882 data_size 0xc(0x4d28 bytes) 2024-04-23T23:50:11.216 INFO:teuthology.orchestra.run.smithi092.stdout:1504 trunc from 0xc00bca1 to 0xdcc5c14 2024-04-23T23:50:11.220 INFO:teuthology.orchestra.run.smithi092.stdout:1505 read 0x361d98a thru 0x3629104 (0xb77b bytes) 2024-04-23T23:50:11.221 INFO:teuthology.orchestra.run.smithi092.stdout:1507 read 0x4128eee thru 0x4129d7c (0xe8f bytes) 2024-04-23T23:50:11.221 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.221 INFO:teuthology.orchestra.run.smithi092.stdout:1509 write 0xc74e700 thru 0xc759d49 (0xb64a bytes) 2024-04-23T23:50:11.224 INFO:teuthology.orchestra.run.smithi092.stdout:1513 read 0x796903a thru 0x796cf5e (0x3f25 bytes) 2024-04-23T23:50:11.224 INFO:teuthology.orchestra.run.smithi092.stdout:1515 writesame 0x2d10f1a thru 0x2d1d0c0 data_size 0x3(0xc1a7 bytes) 2024-04-23T23:50:11.243 INFO:teuthology.orchestra.run.smithi092.stdout:1517 trunc from 0xdcc5c14 to 0x6d5bc76 2024-04-23T23:50:11.253 INFO:teuthology.orchestra.run.smithi092.stdout:1521 write 0x2c32b3c thru 0x2c3386b (0xd30 bytes) 2024-04-23T23:50:11.261 INFO:teuthology.orchestra.run.smithi092.stdout:1522 writesame 0x8fe2219 thru 0x8ff1c10 data_size 0x4c(0xf9f8 bytes) 2024-04-23T23:50:11.263 INFO:teuthology.orchestra.run.smithi092.stdout:1524 write 0x63fe221 thru 0x6407345 (0x9125 bytes) 2024-04-23T23:50:11.265 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.265 INFO:teuthology.orchestra.run.smithi092.stdout:1527 write 0xa98026 thru 0xaa70b0 (0xf08b bytes) 2024-04-23T23:50:11.269 INFO:teuthology.orchestra.run.smithi092.stdout:1528 read 0x53364dc thru 0x534416a (0xdc8f bytes) 2024-04-23T23:50:11.271 INFO:teuthology.orchestra.run.smithi092.stdout:1529 trunc from 0x8ff1c11 to 0x5ac5507 2024-04-23T23:50:11.291 INFO:teuthology.orchestra.run.smithi092.stdout:1531 writesame 0xa7c94a8 thru 0xa7d8351 data_size 0x22(0xeeaa bytes) 2024-04-23T23:50:11.291 INFO:teuthology.orchestra.run.smithi092.stdout:1532 punch from 0x8cb60ba to 0x8cbfd7c, (0x9cc2 bytes) 2024-04-23T23:50:11.291 INFO:teuthology.orchestra.run.smithi092.stdout:1533 read 0x4744097 thru 0x47496ca (0x5634 bytes) 2024-04-23T23:50:11.291 INFO:teuthology.orchestra.run.smithi092.stdout:1534 write 0x4c6cd11 thru 0x4c6f934 (0x2c24 bytes) 2024-04-23T23:50:11.294 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.294 INFO:teuthology.orchestra.run.smithi092.stdout:1536 read 0x422daad thru 0x4237d5d (0xa2b1 bytes) 2024-04-23T23:50:11.294 INFO:teuthology.orchestra.run.smithi092.stdout:1537 write 0x866915a thru 0x866a3be (0x1265 bytes) 2024-04-23T23:50:11.296 INFO:teuthology.orchestra.run.smithi092.stdout:1538 read 0x224b864 thru 0x225447f (0x8c1c bytes) 2024-04-23T23:50:11.296 INFO:teuthology.orchestra.run.smithi092.stdout:1539 writesame 0x63d53d4 thru 0x63e1ab8 data_size 0x1(0xc6e5 bytes) 2024-04-23T23:50:11.305 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:11.319+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:11.323 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:11.323 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:11.323 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:11.323 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:11.330 INFO:teuthology.orchestra.run.smithi092.stdout:1541 trunc from 0xa7d8352 to 0x5aef784 2024-04-23T23:50:11.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:11.343 INFO:teuthology.orchestra.run.smithi092.stdout:1543 write 0x336fdc3 thru 0x337fbff (0xfe3d bytes) 2024-04-23T23:50:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:1544 punch from 0x98c522 to 0x98f54b, (0x3029 bytes) 2024-04-23T23:50:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:1547 punch from 0x4bfc7e4 to 0x4c02003, (0x581f bytes) 2024-04-23T23:50:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:1548 write 0x1bcbfa9 thru 0x1bd9730 (0xd788 bytes) 2024-04-23T23:50:11.346 INFO:teuthology.orchestra.run.smithi092.stdout:1550 punch from 0xbb2afd to 0xbb32fe, (0x801 bytes) 2024-04-23T23:50:11.360 INFO:teuthology.orchestra.run.smithi092.stdout:1552 writesame 0xd14ba06 thru 0xd157202 data_size 0x25(0xb7fd bytes) 2024-04-23T23:50:11.360 INFO:teuthology.orchestra.run.smithi092.stdout:1555 write 0x9634305 thru 0x963f1bd (0xaeb9 bytes) 2024-04-23T23:50:11.363 INFO:teuthology.orchestra.run.smithi092.stdout:1556 punch from 0x9b4e4ac to 0x9b5230a, (0x3e5e bytes) 2024-04-23T23:50:11.363 INFO:teuthology.orchestra.run.smithi092.stdout:1557 writesame 0x1a4c9ca thru 0x1a50cd0 data_size 0x1(0x4307 bytes) 2024-04-23T23:50:11.372 INFO:teuthology.orchestra.run.smithi092.stdout:1558 read 0x8dd4590 thru 0x8dd8c8c (0x46fd bytes) 2024-04-23T23:50:11.372 INFO:teuthology.orchestra.run.smithi092.stdout:1559 read 0x562d9b3 thru 0x5634980 (0x6fce bytes) 2024-04-23T23:50:11.373 INFO:teuthology.orchestra.run.smithi092.stdout:1561 write 0x41b6a66 thru 0x41c0541 (0x9adc bytes) 2024-04-23T23:50:11.376 INFO:teuthology.orchestra.run.smithi092.stdout:1562 trunc from 0xd157203 to 0xcc18f05 2024-04-23T23:50:11.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 3.1e --file /var/log/ceph/exp.3.1e.3 2024-04-23T23:50:11.381 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.381 INFO:teuthology.orchestra.run.smithi092.stdout:1565 write 0x8530d20 thru 0x853b4ff (0xa7e0 bytes) 2024-04-23T23:50:11.390 INFO:teuthology.orchestra.run.smithi092.stdout:1567 writesame 0xd4a5778 thru 0xd4b4b8f data_size 0x92(0xf418 bytes) 2024-04-23T23:50:11.391 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.391 INFO:teuthology.orchestra.run.smithi092.stdout:1569 read 0xb6e1de8 thru 0xb6e792a (0x5b43 bytes) 2024-04-23T23:50:11.391 INFO:teuthology.orchestra.run.smithi092.stdout:1570 punch from 0x50c1aa to 0x50eab0, (0x2906 bytes) 2024-04-23T23:50:11.391 INFO:teuthology.orchestra.run.smithi092.stdout:1571 writesame 0x1f6fb24 thru 0x1f7461c data_size 0x1(0x4af9 bytes) 2024-04-23T23:50:11.402 INFO:teuthology.orchestra.run.smithi092.stdout:1572 write 0x8816323 thru 0x8820e75 (0xab53 bytes) 2024-04-23T23:50:11.405 INFO:teuthology.orchestra.run.smithi092.stdout:1573 write 0xb807ff8 thru 0xb817449 (0xf452 bytes) 2024-04-23T23:50:11.406 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:11.419+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:11.408 INFO:teuthology.orchestra.run.smithi092.stdout:1574 writesame 0x327a731 thru 0x327fb11 data_size 0x1(0x53e1 bytes) 2024-04-23T23:50:11.423 INFO:teuthology.orchestra.run.smithi092.stdout:1575 write 0x1763501 thru 0x176ab7b (0x767b bytes) 2024-04-23T23:50:11.434 INFO:teuthology.orchestra.run.smithi092.stdout:1577 trunc from 0xd4b4b90 to 0x5d6a62f 2024-04-23T23:50:11.461 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:11.461 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:11.461 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:11.461 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:11.461 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:11.462 INFO:teuthology.orchestra.run.smithi092.stdout:1578 write 0xea7772d thru 0xea84c67 (0xd53b bytes) 2024-04-23T23:50:11.464 INFO:teuthology.orchestra.run.smithi092.stdout:1579 read 0x579dd80 thru 0x57a5095 (0x7316 bytes) 2024-04-23T23:50:11.464 INFO:teuthology.orchestra.run.smithi092.stdout:1582 writesame 0x5d0adbf thru 0x5d0fb4b data_size 0x1(0x4d8d bytes) 2024-04-23T23:50:11.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:11.478 INFO:teuthology.orchestra.run.smithi092.stdout:1584 writesame 0xa8ec3b1 thru 0xa8f0a2d data_size 0xf(0x467d bytes) 2024-04-23T23:50:11.482 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.482 INFO:teuthology.orchestra.run.smithi092.stdout:1586 writesame 0xe638316 thru 0xe63c9f3 data_size 0x2(0x46de bytes) 2024-04-23T23:50:11.488 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.489 INFO:teuthology.orchestra.run.smithi092.stdout:1589 trunc from 0xea84c68 to 0xddfb61b 2024-04-23T23:50:11.496 INFO:teuthology.orchestra.run.smithi092.stdout:1591 read 0x2dcbd08 thru 0x2dce110 (0x2409 bytes) 2024-04-23T23:50:11.497 INFO:teuthology.orchestra.run.smithi092.stdout:1592 write 0x501f5c thru 0x503430 (0x14d5 bytes) 2024-04-23T23:50:11.498 INFO:teuthology.orchestra.run.smithi092.stdout:1594 write 0xa2c6732 thru 0xa2ceb0b (0x83da bytes) 2024-04-23T23:50:11.502 INFO:teuthology.orchestra.run.smithi092.stdout:1595 trunc from 0xddfb61b to 0xd54d403 2024-04-23T23:50:11.507 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:11.521+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:11.507 INFO:teuthology.orchestra.run.smithi092.stdout:1596 punch from 0x3eff229 to 0x3f020f1, (0x2ec8 bytes) 2024-04-23T23:50:11.514 INFO:teuthology.orchestra.run.smithi092.stdout:1597 writesame 0xebf40a0 thru 0xebfe491 data_size 0xf(0xa3f2 bytes) 2024-04-23T23:50:11.515 INFO:teuthology.orchestra.run.smithi092.stdout:1599 read 0xe1042a thru 0xe10439 (0x10 bytes) 2024-04-23T23:50:11.515 INFO:teuthology.orchestra.run.smithi092.stdout:1600 write 0x8de20dd thru 0x8ded8c8 (0xb7ec bytes) 2024-04-23T23:50:11.534 INFO:teuthology.orchestra.run.smithi092.stdout:1601 trunc from 0xebfe492 to 0x32f365a 2024-04-23T23:50:11.545 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.545 INFO:teuthology.orchestra.run.smithi092.stdout:1605 writesame 0x17fb696 thru 0x17feb3f data_size 0x3f(0x34aa bytes) 2024-04-23T23:50:11.561 INFO:teuthology.orchestra.run.smithi092.stdout:1608 write 0xa3daf8d thru 0xa3de16f (0x31e3 bytes) 2024-04-23T23:50:11.563 INFO:teuthology.orchestra.run.smithi092.stdout:1609 read 0x2a9ee15 thru 0x2aa35fa (0x47e6 bytes) 2024-04-23T23:50:11.564 INFO:teuthology.orchestra.run.smithi092.stdout:1611 read 0x84232c4 thru 0x8427b85 (0x48c2 bytes) 2024-04-23T23:50:11.564 INFO:teuthology.orchestra.run.smithi092.stdout:1612 write 0x2d43ebd thru 0x2d48df4 (0x4f38 bytes) 2024-04-23T23:50:11.565 INFO:teuthology.orchestra.run.smithi092.stdout:1614 read 0x1044bdc thru 0x1049105 (0x452a bytes) 2024-04-23T23:50:11.566 INFO:teuthology.orchestra.run.smithi092.stdout:1615 write 0x130f5e8 thru 0x1312d1f (0x3738 bytes) 2024-04-23T23:50:11.567 INFO:teuthology.orchestra.run.smithi092.stdout:1616 writesame 0x4b32bb5 thru 0x4b3ee9e data_size 0x3d(0xc2ea bytes) 2024-04-23T23:50:11.568 INFO:teuthology.orchestra.run.smithi092.stdout:1617 punch from 0x3f8c4a8 to 0x3f92fbb, (0x6b13 bytes) 2024-04-23T23:50:11.569 INFO:teuthology.orchestra.run.smithi092.stdout:1619 read 0x1656b85 thru 0x1663d61 (0xd1dd bytes) 2024-04-23T23:50:11.569 INFO:teuthology.orchestra.run.smithi092.stdout:1620 read 0x403948a thru 0x4044946 (0xb4bd bytes) 2024-04-23T23:50:11.569 INFO:teuthology.orchestra.run.smithi092.stdout:1622 punch from 0xa1dc8ce to 0xa1ea5e7, (0xdd19 bytes) 2024-04-23T23:50:11.570 INFO:teuthology.orchestra.run.smithi092.stdout:1623 writesame 0xa59595 thru 0xa694dd data_size 0x1(0xff49 bytes) 2024-04-23T23:50:11.604 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:566 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'3, mlcod=28'1)", 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.701393508000001, 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001175798, 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915528+0000", 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0580000000000001e-06 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915539+0000", 2024-04-23T23:50:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638999999999999e-05 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915554+0000", 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5141e-05 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915565+0000", 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0789e-05 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915620+0000", 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5189999999999998e-05 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915642+0000", 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2262000000000001e-05 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916672+0000", 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010296109999999999 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916703+0000", 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1108000000000001e-05 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1147 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'4, mlcod=28'2)", 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.685309376999999, 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022957910000000002, 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931628+0000", 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6813999999999999e-05 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931636+0000", 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931644+0000", 2024-04-23T23:50:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3359999999999997e-06 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931703+0000", 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9598000000000001e-05 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931724+0000", 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0735000000000001e-05 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933865+0000", 2024-04-23T23:50:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002141165 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933907+0000", 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1768999999999999e-05 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1526 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'16, mlcod=29'14)", 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.23123958, 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021410930000000002, 2024-04-23T23:50:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385682+0000", 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385693+0000", 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0998999999999999e-05 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385710+0000", 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7280000000000001e-05 2024-04-23T23:50:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385719+0000", 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7940000000000008e-06 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385785+0000", 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6109e-05 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385807+0000", 2024-04-23T23:50:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-05 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387778+0000", 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019710769999999999 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387822+0000", 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3665000000000003e-05 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1700 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'19, mlcod=29'17)", 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.044090489, 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090769799999999997, 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572832+0000", 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572856+0000", 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4743e-05 2024-04-23T23:50:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572874+0000", 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7666e-05 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572883+0000", 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4290000000000007e-06 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572946+0000", 2024-04-23T23:50:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000005e-05 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572980+0000", 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4637000000000001e-05 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.573695+0000", 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071479400000000004 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.573738+0000", 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2466999999999998e-05 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1728 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'20, mlcod=29'18)", 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.001361265, 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022012160000000002, 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615560+0000", 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.136e-06 2024-04-23T23:50:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615580+0000", 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9409e-05 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615597+0000", 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.772e-05 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615606+0000", 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6780000000000006e-06 2024-04-23T23:50:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615677+0000", 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0714000000000001e-05 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615701+0000", 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4076000000000001e-05 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617717+0000", 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020162729999999999 2024-04-23T23:50:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617760+0000", 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3210000000000001e-05 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.157358248, 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3876 3.9 e33/22 3:93f02760:::rbd_data.1179c62058ea.0000000000000008:head v 33'13, mlcod=33'10)", 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.1492071769999992, 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014124350000000001, 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467714+0000", 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1100000000000004e-07 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467723+0000", 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0820000000000005e-06 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467738+0000", 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4598e-05 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467746+0000", 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0620000000000001e-06 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467806+0000", 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0591000000000003e-05 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467827+0000", 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0282e-05 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469094+0000", 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001267481 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469126+0000", 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1427999999999998e-05 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4487 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'62, mlcod=32'60)", 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9528825590000007, 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020224290000000001, 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0599999999999999e-07 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664042+0000", 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.501e-06 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664051+0000", 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6609999999999995e-06 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664057+0000", 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6420000000000004e-06 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664109+0000", 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664129+0000", 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0041999999999999e-05 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666021+0000", 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00189174 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666060+0000", 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9149000000000001e-05 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4629 3.0 e33/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 33'4, mlcod=31'2)", 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9374452309999999, 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002138633, 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679476+0000", 2024-04-23T23:50:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1999999999999998e-07 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679583+0000", 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106983 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679599+0000", 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962999999999999e-05 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679608+0000", 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6659999999999997e-06 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679662+0000", 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4385000000000002e-05 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679685+0000", 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2667000000000002e-05 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681580+0000", 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018954270000000001 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681614+0000", 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3722e-05 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5343 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'64, mlcod=33'62)", 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:11.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6934465969999994, 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020278729999999999, 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923480+0000", 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923490+0000", 2024-04-23T23:50:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3919999999999992e-06 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923497+0000", 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923545+0000", 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8647999999999999e-05 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923560+0000", 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5065000000000001e-05 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925471+0000", 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019111340000000001 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925502+0000", 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0321e-05 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4096992880000005, 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:11.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:11.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5602 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'12, mlcod=34'10)", 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.264913+0000", 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3520076739999993, 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000941482, 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264913+0000", 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264913+0000", 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264914+0000", 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.251e-06 2024-04-23T23:50:11.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264947+0000", 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3133999999999998e-05 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264976+0000", 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8534000000000002e-05 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264986+0000", 2024-04-23T23:50:11.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.91e-06 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265074+0000", 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8268000000000003e-05 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265099+0000", 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4828e-05 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265821+0000", 2024-04-23T23:50:11.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072283999999999998 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265854+0000", 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2716999999999999e-05 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.4 epoch 36/22 rep_tid 124 entries 36'70 (0'0) error 3:25e2e575:::rbd_data.117967c411b0.000000000000002e:head by client.4473.0:6553 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'68)", 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5978547699999996, 2024-04-23T23:50:11.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027723230000000001, 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019067+0000", 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9599999999999998e-07 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019079+0000", 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2716e-05 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019107+0000", 2024-04-23T23:50:11.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8232999999999999e-05 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019125+0000", 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717e-05 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019219+0000", 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3455999999999994e-05 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.021838+0000", 2024-04-23T23:50:11.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026193050000000002 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 36/22 rep_tid 127 entries 36'23 (0'0) error 3:05311e05:::rbd_data.117967c411b0.0000000000000034:head by client.4473.0:6561 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'21)", 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5944654739999997, 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002186155, 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022456+0000", 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7899999999999998e-07 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022464+0000", 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022484+0000", 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0369000000000001e-05 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022495+0000", 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1205e-05 2024-04-23T23:50:11.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022559+0000", 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3465000000000003e-05 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.024641+0000", 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020822810000000001 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6580 3.0 e36/22 3:053657a9:::rbd_data.117967c411b0.000000000000003a:head v 36'26, mlcod=36'24)", 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5856029469999999, 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011394210000000001, 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031318+0000", 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-07 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031324+0000", 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031338+0000", 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4036000000000001e-05 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031346+0000", 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2450000000000001e-06 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031392+0000", 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6214000000000002e-05 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031410+0000", 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7859e-05 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032426+0000", 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010161530000000001 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032457+0000", 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1151e-05 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6656 3.4 e36/22 3:23b8cbcf:::rbd_data.117967c411b0.000000000000001f:head v 36'72, mlcod=36'70)", 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5683491849999998, 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001073517, 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:11.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048572+0000", 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048579+0000", 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048593+0000", 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4331000000000001e-05 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048601+0000", 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8539999999999997e-06 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048665+0000", 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3748e-05 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048684+0000", 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9088000000000001e-05 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049604+0000", 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091966300000000003 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049645+0000", 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0936999999999998e-05 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5678299830000002, 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:11.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:11.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:11.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:11.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20374 3.0 e40/22 3:01cd6562:::rbd_data.11793b798a9c.000000000000000e:head v 40'50, mlcod=38'48)", 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.629915695, 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001039091, 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987012+0000", 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9179999999999997e-06 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987022+0000", 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8320000000000001e-06 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987030+0000", 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-04-23T23:50:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987092+0000", 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2902000000000002e-05 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987118+0000", 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5624000000000001e-05 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.988004+0000", 2024-04-23T23:50:11.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088584899999999997 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.988044+0000", 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9869000000000001e-05 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20402 3.9 e40/22 3:94675786:::rbd_data.11793b798a9c.000000000000000b:head v 40'120, mlcod=38'118)", 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:11.052683+0000", 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.56423741299999997, 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000871728, 2024-04-23T23:50:11.683 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052683+0000", 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052683+0000", 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052684+0000", 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052803+0000", 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011900099999999999 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052818+0000", 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4783999999999999e-05 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052825+0000", 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052879+0000", 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3594999999999998e-05 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052896+0000", 2024-04-23T23:50:11.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7374e-05 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.053524+0000", 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062753600000000004 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.053555+0000", 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1143999999999999e-05 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20681 3.9 e40/22 3:94675786:::rbd_data.11793b798a9c.000000000000000b:head v 40'122, mlcod=40'120)", 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:11.422297+0000", 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19462347999999999, 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001011762, 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422297+0000", 2024-04-23T23:50:11.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422297+0000", 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422299+0000", 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.595e-06 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422367+0000", 2024-04-23T23:50:11.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8240999999999999e-05 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422385+0000", 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8199999999999999e-05 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422395+0000", 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.55e-06 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422452+0000", 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7558e-05 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422475+0000", 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3011e-05 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.423263+0000", 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078823999999999995 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:11.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.423309+0000", 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5367000000000001e-05 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:11.692 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:11.694 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:11.695 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.695 INFO:teuthology.orchestra.run.smithi092.stdout:1626 punch from 0x22d5199 to 0x22d5d90, (0xbf7 bytes) 2024-04-23T23:50:11.695 INFO:teuthology.orchestra.run.smithi092.stdout:1629 trunc from 0xa3de170 to 0x154888c 2024-04-23T23:50:11.695 INFO:teuthology.orchestra.run.smithi092.stdout:1631 read 0x6dac2e thru 0x6e9482 (0xe855 bytes) 2024-04-23T23:50:11.695 INFO:teuthology.orchestra.run.smithi092.stdout:1632 writesame 0x7af6501 thru 0x7b003fd data_size 0x1(0x9efd bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1636 write 0xeb82821 thru 0xeb85d09 (0x34e9 bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1639 writesame 0x1abd694 thru 0x1acc65f data_size 0x95(0xefcc bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1640 read 0x3ab8f96 thru 0x3ac223e (0x92a9 bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1641 punch from 0x8c3066a to 0x8c363e2, (0x5d78 bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1643 writesame 0x784cb24 thru 0x7855654 data_size 0x1(0x8b31 bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1644 writesame 0xb6cead2 thru 0xb6cecb8 data_size 0x1(0x1e7 bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1645 punch from 0x42889e3 to 0x429700a, (0xe627 bytes) 2024-04-23T23:50:11.696 INFO:teuthology.orchestra.run.smithi092.stdout:1646 clone 12 order 24 su 65536 sc 13 2024-04-23T23:50:11.697 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:11.707+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:11.795 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:11.808+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:11.799 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:11.799 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:11.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:11.896 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:11.910+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:11.949 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:11.950 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:11.950 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:11.950 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:11.950 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:11.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:12.003 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:12.017+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:12.066 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:12.066 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1143 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.158492990999999, 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017226057999999999, 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.067 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1143 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920898+0000", 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.846e-06 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923748+0000", 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028504649999999999 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923773+0000", 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4311000000000002e-05 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923784+0000", 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1212e-05 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923853+0000", 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9572000000000004e-05 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923965+0000", 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001114 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924349+0000", 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038400400000000002 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937275+0000", 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012926725 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938026+0000", 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075019300000000002 2024-04-23T23:50:12.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938068+0000", 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2809999999999998e-05 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938122+0000", 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3520000000000003e-05 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1142 3.1b e29/22 3:dd409f93:::rbd_data.1179984961fa.0000000000000002:head v 29'17, mlcod=29'15)", 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.155325524, 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014083770000000001, 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924064+0000", 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7000000000000003e-07 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929770+0000", 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057057829999999999 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929783+0000", 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3118999999999999e-05 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929792+0000", 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6549999999999993e-06 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929854+0000", 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2498999999999999e-05 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929871+0000", 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7007000000000002e-05 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938109+0000", 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082374490000000009 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938147+0000", 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8287999999999997e-05 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1148 3.a e29/22 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head v 29'8, mlcod=28'6)", 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.149006683, 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067411720000000001, 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:12.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930383+0000", 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8999999999999995e-07 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930391+0000", 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7209999999999994e-06 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930402+0000", 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0986e-05 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930408+0000", 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.703e-06 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930466+0000", 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7485000000000003e-05 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930485+0000", 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8936e-05 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937077+0000", 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065921360000000002 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937123+0000", 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6415000000000002e-05 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3863 3.e e33/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 33'113, mlcod=33'103)", 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.463373+0000", 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.6160156519999997, 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078337579999999997, 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463373+0000", 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463373+0000", 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463374+0000", 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3200000000000005e-07 2024-04-23T23:50:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463380+0000", 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9610000000000002e-06 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463389+0000", 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5699999999999999e-06 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463397+0000", 2024-04-23T23:50:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4549999999999998e-06 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463467+0000", 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0350000000000002e-05 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463487+0000", 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9922999999999999e-05 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471145+0000", 2024-04-23T23:50:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076580529999999997 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471207+0000", 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1814000000000001e-05 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3864 3.e e33/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 33'114, mlcod=33'103)", 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.464065+0000", 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.6153237770000004, 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071764840000000003, 2024-04-23T23:50:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464065+0000", 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464065+0000", 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464066+0000", 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464071+0000", 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7899999999999999e-06 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464080+0000", 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6809999999999993e-06 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464085+0000", 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7400000000000004e-06 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464131+0000", 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6147999999999999e-05 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464151+0000", 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9986000000000001e-05 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471210+0000", 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070593280000000001 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471242+0000", 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1158999999999998e-05 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7323 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7638711999999996, 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065842770000000004, 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7323 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.01e-07 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315522+0000", 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315527+0000", 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.614e-06 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315532+0000", 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5040000000000004e-06 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.321955+0000", 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006423354 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322022+0000", 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6832e-05 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322102+0000", 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0042000000000004e-05 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7326 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315537+0000", 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7638516219999998, 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067391539999999998, 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7326 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315537+0000", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315537+0000", 2024-04-23T23:50:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315538+0000", 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6199999999999999e-07 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315540+0000", 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9350000000000001e-06 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315545+0000", 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315549+0000", 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3340000000000003e-06 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322127+0000", 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065776949999999997 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322194+0000", 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7348000000000006e-05 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322276+0000", 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2150000000000005e-05 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7327 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7638340729999999, 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068953729999999998, 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7327 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.01e-07 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315557+0000", 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8810000000000001e-06 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315563+0000", 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6910000000000002e-06 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315567+0000", 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5480000000000003e-06 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322301+0000", 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067338060000000002 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322369+0000", 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7972999999999994e-05 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322450+0000", 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1073000000000001e-05 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7330 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7638163489999998, 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070575050000000004, 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7330 2024-04-23T23:50:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9400000000000001e-07 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315575+0000", 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315580+0000", 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315584+0000", 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1030000000000004e-06 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322476+0000", 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068916070000000001 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322547+0000", 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0970999999999995e-05 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322630+0000", 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3077000000000004e-05 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7331 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7637989550000004, 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072241670000000001, 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7331 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315592+0000", 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.796e-06 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315598+0000", 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9549999999999997e-06 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315603+0000", 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6419999999999998e-06 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322656+0000", 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007053719 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322728+0000", 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1250999999999999e-05 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322814+0000", 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6394e-05 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7333 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7637703409999999, 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073801029999999998, 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7333 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.46e-07 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315621+0000", 2024-04-23T23:50:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2390000000000001e-06 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315627+0000", 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-06 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315632+0000", 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8860000000000003e-06 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322843+0000", 2024-04-23T23:50:12.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072113359999999996 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322912+0000", 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8761000000000001e-05 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322999+0000", 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6514999999999994e-05 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7336 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7637507770000003, 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075396389999999999, 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7336 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.96e-07 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315640+0000", 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7400000000000001e-06 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315646+0000", 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.57e-06 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315650+0000", 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3250000000000001e-06 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323025+0000", 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007374551 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323094+0000", 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9546000000000001e-05 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323178+0000", 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3510999999999994e-05 2024-04-23T23:50:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7337 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7637332030000001, 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077025030000000003, 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7337 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6800000000000001e-07 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315658+0000", 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.872e-06 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315664+0000", 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8479999999999997e-06 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315668+0000", 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000003e-06 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323203+0000", 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075353119999999997 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323272+0000", 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8739999999999996e-05 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323358+0000", 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6063000000000006e-05 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7340 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315673+0000", 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7637153760000004, 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078819990000000006, 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7340 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315673+0000", 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315673+0000", 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315674+0000", 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3999999999999997e-07 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315675+0000", 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.66e-06 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315681+0000", 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3789999999999997e-06 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315685+0000", 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5809999999999998e-06 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323397+0000", 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007711412 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323468+0000", 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1229000000000005e-05 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323555+0000", 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7398000000000006e-05 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7341 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7636977570000001, 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080357459999999999, 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7341 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8099999999999998e-07 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315698+0000", 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8970000000000002e-06 2024-04-23T23:50:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315704+0000", 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9309999999999996e-06 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315709+0000", 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5850000000000001e-06 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323583+0000", 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007874097 2024-04-23T23:50:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323648+0000", 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4969999999999996e-05 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323727+0000", 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8885000000000001e-05 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7344 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7636738840000001, 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081910070000000001, 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7344 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315717+0000", 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.813e-06 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315723+0000", 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5790000000000003e-06 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315727+0000", 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5549999999999996e-06 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323754+0000", 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080267580000000002 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323820+0000", 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5456999999999993e-05 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323906+0000", 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6446999999999994e-05 2024-04-23T23:50:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7345 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.7636560240000003, 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083438039999999998, 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7345 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315735+0000", 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.795e-06 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315740+0000", 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315753+0000", 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2265e-05 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323934+0000", 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081817509999999993 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323999+0000", 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4659000000000006e-05 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.324077+0000", 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7509000000000006e-05 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:10306 3.1c e37/22 3:3ce1e20b:::rbd_data.1179129e5779.0000000000000004:head v 37'41, mlcod=37'39)", 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.337445+0000", 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7419437540000002, 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01055997, 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.337445+0000", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.337445+0000", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.337447+0000", 2024-04-23T23:50:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.637e-06 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339521+0000", 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020746240000000002 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339540+0000", 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8187999999999999e-05 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339550+0000", 2024-04-23T23:50:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0835999999999999e-05 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339616+0000", 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5654000000000005e-05 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339634+0000", 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7995999999999998e-05 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.347943+0000", 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083094090000000002 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.348005+0000", 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1625999999999996e-05 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:15000 3.13 e37/22 3:ccff682c:::rbd_data.1179129e5779.0000000000000001:head v 37'32, mlcod=36'30)", 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.572471+0000", 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5069174380000003, 2024-04-23T23:50:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069512910000000001, 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.572471+0000", 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.572471+0000", 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.572472+0000", 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0079999999999999e-06 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573556+0000", 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010837259999999999 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573577+0000", 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0724e-05 2024-04-23T23:50:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573586+0000", 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573650+0000", 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3745999999999996e-05 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573680+0000", 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0788999999999999e-05 2024-04-23T23:50:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579360+0000", 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056797619999999997 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579423+0000", 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2478999999999996e-05 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15281 3.14 3:28b2556a:::rbd_data.1179129e5779.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2048000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.583653+0000", 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4957363460000002, 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0098912140000000006, 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15281 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.583653+0000", 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.583653+0000", 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.583654+0000", 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8080000000000001e-06 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585797+0000", 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021424090000000001 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585813+0000", 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6249999999999999e-05 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585820+0000", 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8329999999999996e-06 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585886+0000", 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6019999999999995e-05 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.586025+0000", 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000138968 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.586308+0000", 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028303699999999998 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593123+0000", 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006814886 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593468+0000", 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00034489200000000002 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593513+0000", 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4876000000000002e-05 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593544+0000", 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1235000000000002e-05 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:12.157 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:12.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:12.159 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:12.172+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:12.172 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-04-23T23:50:12.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:12.259 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:12.273+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:12.304 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:12.305 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:12.305 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:12.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:12.342 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:12.342 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:12.347 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.347 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.347 INFO:teuthology.orchestra.run.smithi092.stdout:1649 punch from 0xd281459 to 0xd28dd8d, (0xc934 bytes) 2024-04-23T23:50:12.350 INFO:teuthology.orchestra.run.smithi092.stdout:1652 write 0x32b8334 thru 0x32b996c (0x1639 bytes) 2024-04-23T23:50:12.359 INFO:teuthology.orchestra.run.smithi092.stdout:1653 read 0xaafc26f thru 0xab0a751 (0xe4e3 bytes) 2024-04-23T23:50:12.359 INFO:teuthology.orchestra.run.smithi092.stdout:1654 writesame 0xb9ca8a0 thru 0xb9d3ac7 data_size 0xc(0x9228 bytes) 2024-04-23T23:50:12.365 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:12.365 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:12.373 INFO:teuthology.orchestra.run.smithi092.stdout:1655 writesame 0x3c3d178 thru 0x3c44fbc data_size 0x19(0x7e45 bytes) 2024-04-23T23:50:12.386 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:12.387 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:12.387 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:12.387 INFO:teuthology.orchestra.run.smithi092.stdout:1656 read 0x6383ada thru 0x639096e (0xce95 bytes) 2024-04-23T23:50:12.387 INFO:teuthology.orchestra.run.smithi092.stdout:1658 read 0x4c6a226 thru 0x4c756e1 (0xb4bc bytes) 2024-04-23T23:50:12.387 INFO:teuthology.orchestra.run.smithi092.stdout:1659 write 0xe8eb453 thru 0xe8f78f6 (0xc4a4 bytes) 2024-04-23T23:50:12.395 INFO:teuthology.orchestra.run.smithi092.stdout:1660 read 0x38c25d thru 0x38e3ce (0x2172 bytes) 2024-04-23T23:50:12.397 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:12.411+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:12.405 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:12.405 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:12.408 INFO:teuthology.orchestra.run.smithi092.stdout:1661 trunc from 0xeb85d0a to 0x3f71993 2024-04-23T23:50:12.431 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:12.454 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:12.460 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:12.474+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:12.468 INFO:teuthology.orchestra.run.smithi092.stdout:1662 punch from 0x1c6f109 to 0x1c749ff, (0x58f6 bytes) 2024-04-23T23:50:12.468 INFO:teuthology.orchestra.run.smithi092.stdout:1663 punch from 0xe3da19 to 0xe44287, (0x686e bytes) 2024-04-23T23:50:12.469 INFO:teuthology.orchestra.run.smithi092.stdout:1664 punch from 0xf858ba to 0xf94590, (0xecd6 bytes) 2024-04-23T23:50:12.474 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:12.478 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:12.478 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:12.484 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:12.484 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:12.485 INFO:teuthology.orchestra.run.smithi092.stdout:1665 write 0xcdea73c thru 0xcdeab66 (0x42b bytes) 2024-04-23T23:50:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:Exporting 3.1e info 3.1e( v 37'112 (0'0,37'112] local-lis/les=22/23 n=3 ec=22/22 lis/c=22/22 les/c/f=23/23/0 sis=22) 2024-04-23T23:50:12.493 INFO:teuthology.orchestra.run.smithi092.stdout:1666 trunc from 0xcdeab67 to 0x7074360 2024-04-23T23:50:12.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:Read #3:7902102c:::rbd_object_map.117916e6274d:head# 2024-04-23T23:50:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:Read #3:7a32bd15:::rbd_object_map.1179b25581a8:head# 2024-04-23T23:50:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:Read #3:7c5de93e:::rbd_header.11798eaf3f6b:head# 2024-04-23T23:50:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:Export successful 2024-04-23T23:50:12.501 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.513 INFO:teuthology.orchestra.run.smithi092.stdout:1670 write 0xcaedf88 thru 0xcaf77b6 (0x982f bytes) 2024-04-23T23:50:12.514 INFO:teuthology.orchestra.run.smithi092.stdout:1671 read 0xbf79951 thru 0xbf7c519 (0x2bc9 bytes) 2024-04-23T23:50:12.515 INFO:teuthology.orchestra.run.smithi092.stdout:1672 write 0x84813ed thru 0x8489740 (0x8354 bytes) 2024-04-23T23:50:12.516 INFO:teuthology.orchestra.run.smithi092.stdout:1674 write 0x1cee896 thru 0x1cfc978 (0xe0e3 bytes) 2024-04-23T23:50:12.518 INFO:teuthology.orchestra.run.smithi092.stdout:1675 write 0x994ea2e thru 0x99567fc (0x7dcf bytes) 2024-04-23T23:50:12.519 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.519 INFO:teuthology.orchestra.run.smithi092.stdout:1679 write 0x7d43920 thru 0x7d51a17 (0xe0f8 bytes) 2024-04-23T23:50:12.520 INFO:teuthology.orchestra.run.smithi092.stdout:1680 read 0x3152bb6 thru 0x316148f (0xe8da bytes) 2024-04-23T23:50:12.521 INFO:teuthology.orchestra.run.smithi092.stdout:1682 read 0xcd613d thru 0xcd88b5 (0x2779 bytes) 2024-04-23T23:50:12.525 INFO:teuthology.orchestra.run.smithi092.stdout:1683 trunc from 0xcaf77b7 to 0x9115950 2024-04-23T23:50:12.537 INFO:teuthology.orchestra.run.smithi092.stdout:1684 trunc from 0x9115950 to 0x4e0a8ad 2024-04-23T23:50:12.546 INFO:teuthology.orchestra.run.smithi092.stdout:1685 read 0x49ddaf1 thru 0x49eaa2b (0xcf3b bytes) 2024-04-23T23:50:12.546 INFO:teuthology.orchestra.run.smithi092.stdout:1688 trunc from 0x4e0a8ad to 0x4f7b71c 2024-04-23T23:50:12.550 INFO:teuthology.orchestra.run.smithi092.stdout:1689 read 0x2b2c9e4 thru 0x2b30f0b (0x4528 bytes) 2024-04-23T23:50:12.556 INFO:teuthology.orchestra.run.smithi092.stdout:1690 trunc from 0x4f7b71c to 0x92e93b9 2024-04-23T23:50:12.561 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:12.575+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:12.569 INFO:teuthology.orchestra.run.smithi092.stdout:1693 write 0xb8d18b1 thru 0xb8db047 (0x9797 bytes) 2024-04-23T23:50:12.570 INFO:teuthology.orchestra.run.smithi092.stdout:1694 writesame 0x460da0e thru 0x460f7b1 data_size 0xe(0x1da4 bytes) 2024-04-23T23:50:12.572 INFO:teuthology.orchestra.run.smithi092.stdout:1696 read 0x5330335 thru 0x53390f2 (0x8dbe bytes) 2024-04-23T23:50:12.572 INFO:teuthology.orchestra.run.smithi092.stdout:1698 punch from 0x45eab98 to 0x45ec19a, (0x1602 bytes) 2024-04-23T23:50:12.573 INFO:teuthology.orchestra.run.smithi092.stdout:1700 write 0x33581e2 thru 0x3365358 (0xd177 bytes) 2024-04-23T23:50:12.574 INFO:teuthology.orchestra.run.smithi092.stdout:1702 read 0x2769ea1 thru 0x2771e60 (0x7fc0 bytes) 2024-04-23T23:50:12.578 INFO:teuthology.orchestra.run.smithi092.stdout:1703 trunc from 0xb8db048 to 0xe53d0c8 2024-04-23T23:50:12.582 INFO:teuthology.orchestra.run.smithi092.stdout:1705 write 0xdfad17b thru 0xdfb64e4 (0x936a bytes) 2024-04-23T23:50:12.586 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.586 INFO:teuthology.orchestra.run.smithi092.stdout:1708 writesame 0x8700ed6 thru 0x870ac4a data_size 0x1(0x9d75 bytes) 2024-04-23T23:50:12.610 INFO:teuthology.orchestra.run.smithi092.stdout:1710 punch from 0x4c6e1be to 0x4c77957, (0x9799 bytes) 2024-04-23T23:50:12.610 INFO:teuthology.orchestra.run.smithi092.stdout:1711 write 0x304b64b thru 0x30576a1 (0xc057 bytes) 2024-04-23T23:50:12.613 INFO:teuthology.orchestra.run.smithi092.stdout:1713 read 0x85a978c thru 0x85b2673 (0x8ee8 bytes) 2024-04-23T23:50:12.613 INFO:teuthology.orchestra.run.smithi092.stdout:1714 writesame 0xcd260c2 thru 0xcd26953 data_size 0x1(0x892 bytes) 2024-04-23T23:50:12.616 INFO:teuthology.orchestra.run.smithi092.stdout:1715 writesame 0x455cd66 thru 0x456a02f data_size 0x1(0xd2ca bytes) 2024-04-23T23:50:12.640 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:12.640 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:12.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:12.650 INFO:teuthology.orchestra.run.smithi092.stdout:1716 write 0x4b5922a thru 0x4b5a630 (0x1407 bytes) 2024-04-23T23:50:12.651 INFO:teuthology.orchestra.run.smithi092.stdout:1717 write 0x951b01d thru 0x9521ee9 (0x6ecd bytes) 2024-04-23T23:50:12.654 INFO:teuthology.orchestra.run.smithi092.stdout:1718 write 0x853e6ef thru 0x854cee8 (0xe7fa bytes) 2024-04-23T23:50:12.670 INFO:teuthology.orchestra.run.smithi092.stdout:1719 trunc from 0xe53d0c8 to 0x1566502 2024-04-23T23:50:12.693 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:12.707+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:12.700 INFO:teuthology.orchestra.run.smithi092.stdout:1720 writesame 0xb61c152 thru 0xb61db32 data_size 0x5(0x19e1 bytes) 2024-04-23T23:50:12.701 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.702 INFO:teuthology.orchestra.run.smithi092.stdout:1724 write 0xb4c0a8f thru 0xb4ca531 (0x9aa3 bytes) 2024-04-23T23:50:12.703 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.703 INFO:teuthology.orchestra.run.smithi092.stdout:1726 write 0x8448ade thru 0x844dd71 (0x5294 bytes) 2024-04-23T23:50:12.704 INFO:teuthology.orchestra.run.smithi092.stdout:1727 read 0x338c8bb thru 0x338ceea (0x630 bytes) 2024-04-23T23:50:12.705 INFO:teuthology.orchestra.run.smithi092.stdout:1728 write 0x9a5dd8b thru 0x9a67ef9 (0xa16f bytes) 2024-04-23T23:50:12.706 INFO:teuthology.orchestra.run.smithi092.stdout:1729 read 0x2967e63 thru 0x296fd6e (0x7f0c bytes) 2024-04-23T23:50:12.706 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.706 INFO:teuthology.orchestra.run.smithi092.stdout:1731 read 0xa80657d thru 0xa8065bf (0x43 bytes) 2024-04-23T23:50:12.707 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.707 INFO:teuthology.orchestra.run.smithi092.stdout:1735 read 0x67882db thru 0x6789f46 (0x1c6c bytes) 2024-04-23T23:50:12.707 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.715 INFO:teuthology.orchestra.run.smithi092.stdout:1737 write 0xd83c968 thru 0xd848385 (0xba1e bytes) 2024-04-23T23:50:12.730 INFO:teuthology.orchestra.run.smithi092.stdout:1738 trunc from 0xd848386 to 0x307694f 2024-04-23T23:50:12.740 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.740 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.740 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.758 INFO:teuthology.orchestra.run.smithi092.stdout:1745 writesame 0xd213509 thru 0xd215fdf data_size 0x1(0x2ad7 bytes) 2024-04-23T23:50:12.759 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:12.759 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:12.763 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:12.776+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:12.766 INFO:teuthology.orchestra.run.smithi092.stdout:1747 write 0xe0bf19b thru 0xe0ccdf2 (0xdc58 bytes) 2024-04-23T23:50:12.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:12.769 INFO:teuthology.orchestra.run.smithi092.stdout:1748 read 0x7f3cd2d thru 0x7f40959 (0x3c2d bytes) 2024-04-23T23:50:12.770 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.770 INFO:teuthology.orchestra.run.smithi092.stdout:1750 read 0xaa7e377 thru 0xaa87ce6 (0x9970 bytes) 2024-04-23T23:50:12.770 INFO:teuthology.orchestra.run.smithi092.stdout:1751 write 0xd12b8d1 thru 0xd136c8a (0xb3ba bytes) 2024-04-23T23:50:12.773 INFO:teuthology.orchestra.run.smithi092.stdout:1753 punch from 0x9994156 to 0x9995b00, (0x19aa bytes) 2024-04-23T23:50:12.773 INFO:teuthology.orchestra.run.smithi092.stdout:1754 writesame 0x2bdf88d thru 0x2be2d9c data_size 0x18(0x3510 bytes) 2024-04-23T23:50:12.775 INFO:teuthology.orchestra.run.smithi092.stdout:1755 punch from 0x195d211 to 0x196b522, (0xe311 bytes) 2024-04-23T23:50:12.775 INFO:teuthology.orchestra.run.smithi092.stdout:1756 punch from 0xd1ad308 to 0xd1b6d64, (0x9a5c bytes) 2024-04-23T23:50:12.775 INFO:teuthology.orchestra.run.smithi092.stdout:1757 read 0xa98b761 thru 0xa98c63d (0xedd bytes) 2024-04-23T23:50:12.775 INFO:teuthology.orchestra.run.smithi092.stdout:1758 punch from 0x141dc2f to 0x141de60, (0x231 bytes) 2024-04-23T23:50:12.775 INFO:teuthology.orchestra.run.smithi092.stdout:1759 punch from 0x4754096 to 0x4760d9e, (0xcd08 bytes) 2024-04-23T23:50:12.789 INFO:teuthology.orchestra.run.smithi092.stdout:1760 trunc from 0xe0ccdf3 to 0x235dfa9 2024-04-23T23:50:12.800 INFO:teuthology.orchestra.run.smithi092.stdout:1762 write 0xf8b04e thru 0xf8c934 (0x18e7 bytes) 2024-04-23T23:50:12.800 INFO:teuthology.orchestra.run.smithi092.stdout:1763 punch from 0x153c913 to 0x1540b37, (0x4224 bytes) 2024-04-23T23:50:12.801 INFO:teuthology.orchestra.run.smithi092.stdout:1764 trunc from 0x235dfa9 to 0x1a19930 2024-04-23T23:50:12.808 INFO:teuthology.orchestra.run.smithi092.stdout:1765 writesame 0xa51255 thru 0xa512ca data_size 0x1(0x76 bytes) 2024-04-23T23:50:12.809 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.809 INFO:teuthology.orchestra.run.smithi092.stdout:1768 read 0x10a9090 thru 0x10af148 (0x60b9 bytes) 2024-04-23T23:50:12.811 INFO:teuthology.orchestra.run.smithi092.stdout:1769 trunc from 0x1a19930 to 0x2c72d33 2024-04-23T23:50:12.830 INFO:teuthology.orchestra.run.smithi092.stdout:1770 write 0x973a60b thru 0x9746abb (0xc4b1 bytes) 2024-04-23T23:50:12.832 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.832 INFO:teuthology.orchestra.run.smithi092.stdout:1772 punch from 0x8c41e58 to 0x8c490ae, (0x7256 bytes) 2024-04-23T23:50:12.832 INFO:teuthology.orchestra.run.smithi092.stdout:1773 writesame 0x2830246 thru 0x2836817 data_size 0x1(0x65d2 bytes) 2024-04-23T23:50:12.849 INFO:teuthology.orchestra.run.smithi092.stdout:1774 trunc from 0x9746abc to 0x87843cc 2024-04-23T23:50:12.857 INFO:teuthology.orchestra.run.smithi092.stdout:1775 write 0x2723293 thru 0x27325b8 (0xf326 bytes) 2024-04-23T23:50:12.859 INFO:teuthology.orchestra.run.smithi092.stdout:1776 read 0x5bf5f9f thru 0x5bfd2b4 (0x7316 bytes) 2024-04-23T23:50:12.864 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:12.878+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:12.866 INFO:teuthology.orchestra.run.smithi092.stdout:1777 writesame 0xa3c8ec4 thru 0xa3cb894 data_size 0x1(0x29d1 bytes) 2024-04-23T23:50:12.867 INFO:teuthology.orchestra.run.smithi092.stdout:1779 write 0x46413ff thru 0x46428d4 (0x14d6 bytes) 2024-04-23T23:50:12.869 INFO:teuthology.orchestra.run.smithi092.stdout:1780 read 0x9d3ade6 thru 0x9d40906 (0x5b21 bytes) 2024-04-23T23:50:12.869 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.870 INFO:teuthology.orchestra.run.smithi092.stdout:1782 punch from 0x9d247d2 to 0x9d2b0a4, (0x68d2 bytes) 2024-04-23T23:50:12.870 INFO:teuthology.orchestra.run.smithi092.stdout:1783 punch from 0xa3c74d2 to 0xa3c7d16, (0x844 bytes) 2024-04-23T23:50:12.870 INFO:teuthology.orchestra.run.smithi092.stdout:1784 write 0x4f1e918 thru 0x4f211b6 (0x289f bytes) 2024-04-23T23:50:12.871 INFO:teuthology.orchestra.run.smithi092.stdout:1785 write 0x632df65 thru 0x633c3ea (0xe486 bytes) 2024-04-23T23:50:12.872 INFO:teuthology.orchestra.run.smithi092.stdout:1786 read 0x5aedf46 thru 0x5afb2d7 (0xd392 bytes) 2024-04-23T23:50:12.872 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.872 INFO:teuthology.orchestra.run.smithi092.stdout:1788 write 0x70dd031 thru 0x70eb160 (0xe130 bytes) 2024-04-23T23:50:12.874 INFO:teuthology.orchestra.run.smithi092.stdout:1789 read 0x3b81b81 thru 0x3b88133 (0x65b3 bytes) 2024-04-23T23:50:12.874 INFO:teuthology.orchestra.run.smithi092.stdout:1790 writesame 0x9b8922b thru 0x9b96d22 data_size 0x4d(0xdaf8 bytes) 2024-04-23T23:50:12.876 INFO:teuthology.orchestra.run.smithi092.stdout:1792 read 0x39fb987 thru 0x3a0acb8 (0xf332 bytes) 2024-04-23T23:50:12.883 INFO:teuthology.orchestra.run.smithi092.stdout:1793 write 0xb5f5766 thru 0xb5fda21 (0x82bc bytes) 2024-04-23T23:50:12.885 INFO:teuthology.orchestra.run.smithi092.stdout:1794 read 0x6573288 thru 0x657d374 (0xa0ed bytes) 2024-04-23T23:50:12.885 INFO:teuthology.orchestra.run.smithi092.stdout:1795 writesame 0x20613f0 thru 0x206b137 data_size 0x1c(0x9d48 bytes) 2024-04-23T23:50:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:1797 read 0x8f6f058 thru 0x8f7f013 (0xffbc bytes) 2024-04-23T23:50:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:1798 writesame 0xa8bc980 thru 0xa8bca42 data_size 0x1(0xc3 bytes) 2024-04-23T23:50:12.889 INFO:teuthology.orchestra.run.smithi092.stdout:1799 punch from 0x78b30e9 to 0x78c2582, (0xf499 bytes) 2024-04-23T23:50:12.889 INFO:teuthology.orchestra.run.smithi092.stdout:1800 write 0x2ac973b thru 0x2ad0ee1 (0x77a7 bytes) 2024-04-23T23:50:12.897 INFO:teuthology.orchestra.run.smithi092.stdout:1801 trunc from 0xb5fda22 to 0x593496f 2024-04-23T23:50:12.904 INFO:teuthology.orchestra.run.smithi092.stdout:1802 read 0x13b2a0 thru 0x147a56 (0xc7b7 bytes) 2024-04-23T23:50:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:1803 punch from 0x21bbc5f to 0x21c8ecd, (0xd26e bytes) 2024-04-23T23:50:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:1804 punch from 0x7b37ca to 0x7bfb47, (0xc37d bytes) 2024-04-23T23:50:12.906 INFO:teuthology.orchestra.run.smithi092.stdout:1806 flatten 2024-04-23T23:50:12.922 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:12.922 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:12.923 INFO:teuthology.orchestra.run.smithi092.stdout:1808 writesame 0xa5f4467 thru 0xa5ff871 data_size 0x1(0xb40b bytes) 2024-04-23T23:50:12.924 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:12.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:12.934 INFO:teuthology.orchestra.run.smithi092.stdout:1810 write 0xe4281d0 thru 0xe42dea1 (0x5cd2 bytes) 2024-04-23T23:50:12.936 INFO:teuthology.orchestra.run.smithi092.stdout:1811 punch from 0x393fd5f to 0x3944eab, (0x514c bytes) 2024-04-23T23:50:12.937 INFO:teuthology.orchestra.run.smithi092.stdout:1812 punch from 0xa7f0eac to 0xa7fc9c4, (0xbb18 bytes) 2024-04-23T23:50:12.937 INFO:teuthology.orchestra.run.smithi092.stdout:1814 writesame 0x4a7323c thru 0x4a76c77 data_size 0x2(0x3a3c bytes) 2024-04-23T23:50:12.941 INFO:teuthology.orchestra.run.smithi092.stdout:1815 read 0xa2a3bfb thru 0xa2ad9c8 (0x9dce bytes) 2024-04-23T23:50:12.942 INFO:teuthology.orchestra.run.smithi092.stdout:1816 write 0x52b7c05 thru 0x52bc30f (0x470b bytes) 2024-04-23T23:50:12.943 INFO:teuthology.orchestra.run.smithi092.stdout:1817 write 0xd16a919 thru 0xd17693b (0xc023 bytes) 2024-04-23T23:50:12.945 INFO:teuthology.orchestra.run.smithi092.stdout:1818 read 0x2cb21aa thru 0x2cbd867 (0xb6be bytes) 2024-04-23T23:50:12.946 INFO:teuthology.orchestra.run.smithi092.stdout:1819 read 0x1406e73 thru 0x140f3cd (0x855b bytes) 2024-04-23T23:50:12.950 INFO:teuthology.orchestra.run.smithi092.stdout:1820 trunc from 0xe42dea2 to 0xb2144e4 2024-04-23T23:50:12.960 INFO:teuthology.orchestra.run.smithi092.stdout:1821 writesame 0x6770143 thru 0x677e9db data_size 0x1(0xe899 bytes) 2024-04-23T23:50:12.972 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:12.986+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:12.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:50:12.996 INFO:teuthology.orchestra.run.smithi092.stdout:1822 write 0x9a7c4 thru 0xa32c5 (0x8b02 bytes) 2024-04-23T23:50:12.998 INFO:teuthology.orchestra.run.smithi092.stdout:1823 punch from 0xa9695a6 to 0xa973ab1, (0xa50b bytes) 2024-04-23T23:50:12.998 INFO:teuthology.orchestra.run.smithi092.stdout:1824 read 0xea5d04 thru 0xead1e3 (0x74e0 bytes) 2024-04-23T23:50:12.999 INFO:teuthology.orchestra.run.smithi092.stdout:1826 punch from 0x31bdf15 to 0x31c008d, (0x2178 bytes) 2024-04-23T23:50:13.008 INFO:teuthology.orchestra.run.smithi092.stdout:1828 writesame 0xec9bc66 thru 0xeca505b data_size 0x76(0x93f6 bytes) 2024-04-23T23:50:13.008 INFO:teuthology.orchestra.run.smithi092.stdout:1829 write 0x1e72d21 thru 0x1e801f5 (0xd4d5 bytes) 2024-04-23T23:50:13.010 INFO:teuthology.orchestra.run.smithi092.stdout:1832 read 0x63155e1 thru 0x6319290 (0x3cb0 bytes) 2024-04-23T23:50:13.010 INFO:teuthology.orchestra.run.smithi092.stdout:1833 read 0xa86261d thru 0xa86ff70 (0xd954 bytes) 2024-04-23T23:50:13.017 INFO:teuthology.orchestra.run.smithi092.stdout:1834 trunc from 0xeca505c to 0x9a1fe12 2024-04-23T23:50:13.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 3.1e 2024-04-23T23:50:13.030 INFO:teuthology.orchestra.run.smithi092.stdout:1836 trunc from 0x9a1fe12 to 0xbbb2f1f 2024-04-23T23:50:13.048 INFO:teuthology.orchestra.run.smithi092.stdout:1837 trunc from 0xbbb2f1f to 0xd128d8 2024-04-23T23:50:13.065 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:13.079+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:13.074 INFO:teuthology.orchestra.run.smithi092.stdout:1838 write 0x8b3c770 thru 0x8b476a8 (0xaf39 bytes) 2024-04-23T23:50:13.075 INFO:teuthology.orchestra.run.smithi092.stdout:1839 read 0x6a1cade thru 0x6a23985 (0x6ea8 bytes) 2024-04-23T23:50:13.076 INFO:teuthology.orchestra.run.smithi092.stdout:1840 punch from 0x402d22 to 0x40f12c, (0xc40a bytes) 2024-04-23T23:50:13.076 INFO:teuthology.orchestra.run.smithi092.stdout:1841 punch from 0x67c58ff to 0x67c66e1, (0xde2 bytes) 2024-04-23T23:50:13.079 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.079 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.083 INFO:teuthology.orchestra.run.smithi092.stdout:1843 trunc from 0x8b476a9 to 0x28957d3 2024-04-23T23:50:13.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:13.110 INFO:teuthology.orchestra.run.smithi092.stdout:1845 write 0xa9e9003 thru 0xa9f1bc0 (0x8bbe bytes) 2024-04-23T23:50:13.110 INFO:teuthology.orchestra.run.smithi092.stdout:1847 read 0x393b1ef thru 0x394b056 (0xfe68 bytes) 2024-04-23T23:50:13.111 INFO:teuthology.orchestra.run.smithi092.stdout:1848 write 0xd81ad2 thru 0xd875d5 (0x5b04 bytes) 2024-04-23T23:50:13.112 INFO:teuthology.orchestra.run.smithi092.stdout:1849 punch from 0x42524f1 to 0x425d68e, (0xb19d bytes) 2024-04-23T23:50:13.112 INFO:teuthology.orchestra.run.smithi092.stdout:1850 write 0x91cd16e thru 0x91d83d5 (0xb268 bytes) 2024-04-23T23:50:13.114 INFO:teuthology.orchestra.run.smithi092.stdout:1852 write 0xa1bdb8c thru 0xa1bec82 (0x10f7 bytes) 2024-04-23T23:50:13.121 INFO:teuthology.orchestra.run.smithi092.stdout:1854 write 0xbd57436 thru 0xbd6603e (0xec09 bytes) 2024-04-23T23:50:13.122 INFO:teuthology.orchestra.run.smithi092.stdout:1855 read 0xb037e8 thru 0xb0a3a3 (0x6bbc bytes) 2024-04-23T23:50:13.122 INFO:teuthology.orchestra.run.smithi092.stdout:1857 write 0x1646218 thru 0x16482d6 (0x20bf bytes) 2024-04-23T23:50:13.123 INFO:teuthology.orchestra.run.smithi092.stdout:1859 writesame 0x21865c5 thru 0x218d538 data_size 0xe(0x6f74 bytes) 2024-04-23T23:50:13.125 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.126 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.133 INFO:teuthology.orchestra.run.smithi092.stdout:1862 trunc from 0xbd6603f to 0x581845e 2024-04-23T23:50:13.142 INFO:teuthology.orchestra.run.smithi092.stdout:1863 read 0x504b90c thru 0x504e41a (0x2b0f bytes) 2024-04-23T23:50:13.151 INFO:teuthology.orchestra.run.smithi092.stdout:1864 write 0x83ae6fe thru 0x83b0537 (0x1e3a bytes) 2024-04-23T23:50:13.152 INFO:teuthology.orchestra.run.smithi092.stdout:1865 read 0xb3853f thru 0xb3df71 (0x5a33 bytes) 2024-04-23T23:50:13.165 INFO:teuthology.orchestra.run.smithi092.stdout:1867 write 0xe8aace9 thru 0xe8b359f (0x88b7 bytes) 2024-04-23T23:50:13.166 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:13.180+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:13.168 INFO:teuthology.orchestra.run.smithi092.stdout:1869 write 0xe47b506 thru 0xe47e2f0 (0x2deb bytes) 2024-04-23T23:50:13.170 INFO:teuthology.orchestra.run.smithi092.stdout:1870 read 0xbf48af9 thru 0xbf524dd (0x99e5 bytes) 2024-04-23T23:50:13.182 INFO:teuthology.orchestra.run.smithi092.stdout:1871 trunc from 0xe8b35a0 to 0x59fd875 2024-04-23T23:50:13.199 INFO:teuthology.orchestra.run.smithi092.stdout:1872 read 0x3dfeac thru 0x3e8f09 (0x905e bytes) 2024-04-23T23:50:13.199 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.206 INFO:teuthology.orchestra.run.smithi092.stdout:1874 trunc from 0x59fd875 to 0xb75e857 2024-04-23T23:50:13.211 INFO:teuthology.orchestra.run.smithi092.stdout:1875 write 0x420cccb thru 0x420d907 (0xc3d bytes) 2024-04-23T23:50:13.212 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.212 INFO:teuthology.orchestra.run.smithi092.stdout:1880 write 0xabeee8 thru 0xacd3c9 (0xe4e2 bytes) 2024-04-23T23:50:13.214 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.214 INFO:teuthology.orchestra.run.smithi092.stdout:1884 write 0x9d9e7f9 thru 0x9da9a76 (0xb27e bytes) 2024-04-23T23:50:13.215 INFO:teuthology.orchestra.run.smithi092.stdout:1887 punch from 0x91aab7 to 0x923b19, (0x9062 bytes) 2024-04-23T23:50:13.215 INFO:teuthology.orchestra.run.smithi092.stdout:1888 write 0x8160416 thru 0x816cf58 (0xcb43 bytes) 2024-04-23T23:50:13.216 INFO:teuthology.orchestra.run.smithi092.stdout:1889 write 0x67b24a8 thru 0x67c2292 (0xfdeb bytes) 2024-04-23T23:50:13.218 INFO:teuthology.orchestra.run.smithi092.stdout:1890 punch from 0x3c61144 to 0x3c6feca, (0xed86 bytes) 2024-04-23T23:50:13.218 INFO:teuthology.orchestra.run.smithi092.stdout:1892 read 0x3ef8110 thru 0x3efe7fd (0x66ee bytes) 2024-04-23T23:50:13.218 INFO:teuthology.orchestra.run.smithi092.stdout:1893 read 0x33d6dc5 thru 0x33e1c09 (0xae45 bytes) 2024-04-23T23:50:13.219 INFO:teuthology.orchestra.run.smithi092.stdout:1896 read 0x40aebfb thru 0x40aee9b (0x2a1 bytes) 2024-04-23T23:50:13.219 INFO:teuthology.orchestra.run.smithi092.stdout:1897 read 0x1c72ddc thru 0x1c7aded (0x8012 bytes) 2024-04-23T23:50:13.219 INFO:teuthology.orchestra.run.smithi092.stdout:1898 write 0x47bb3c5 thru 0x47c5b16 (0xa752 bytes) 2024-04-23T23:50:13.221 INFO:teuthology.orchestra.run.smithi092.stdout:1899 write 0x67147a3 thru 0x671e6c0 (0x9f1e bytes) 2024-04-23T23:50:13.222 INFO:teuthology.orchestra.run.smithi092.stdout:1900 write 0x4bfd84 thru 0x4c9f0c (0xa189 bytes) 2024-04-23T23:50:13.224 INFO:teuthology.orchestra.run.smithi092.stdout:1901 punch from 0x8f7473d to 0x8f7aab0, (0x6373 bytes) 2024-04-23T23:50:13.230 INFO:teuthology.orchestra.run.smithi092.stdout:1902 write 0xc2dd3f2 thru 0xc2e10c6 (0x3cd5 bytes) 2024-04-23T23:50:13.236 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.236 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.237 INFO:teuthology.orchestra.run.smithi092.stdout:1908 trunc from 0xc2e10c7 to 0x7a3028e 2024-04-23T23:50:13.244 INFO:teuthology.orchestra.run.smithi092.stdout:1909 read 0x4e6c7ed thru 0x4e6cadd (0x2f1 bytes) 2024-04-23T23:50:13.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:13.248 INFO:teuthology.orchestra.run.smithi092.stdout:1910 trunc from 0x7a3028e to 0x478f0c4 2024-04-23T23:50:13.255 INFO:teuthology.orchestra.run.smithi092.stdout:1911 read 0x21c3a94 thru 0x21c8957 (0x4ec4 bytes) 2024-04-23T23:50:13.255 INFO:teuthology.orchestra.run.smithi092.stdout:1912 punch from 0x4371853 to 0x4375146, (0x38f3 bytes) 2024-04-23T23:50:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:1913 trunc from 0x478f0c4 to 0x7457228 2024-04-23T23:50:13.263 INFO:teuthology.orchestra.run.smithi092.stdout:1915 read 0x5e6cc6d thru 0x5e7933b (0xc6cf bytes) 2024-04-23T23:50:13.267 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:13.281+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:13.269 INFO:teuthology.orchestra.run.smithi092.stdout:1916 trunc from 0x7457228 to 0xb3b5408 2024-04-23T23:50:13.282 INFO:teuthology.orchestra.run.smithi092.stdout:1920 writesame 0xe0a875b thru 0xe0b1fdb data_size 0x1(0x9881 bytes) 2024-04-23T23:50:13.282 INFO:teuthology.orchestra.run.smithi092.stdout:1921 read 0x644040f thru 0x644cf42 (0xcb34 bytes) 2024-04-23T23:50:13.282 INFO:teuthology.orchestra.run.smithi092.stdout:1922 writesame 0x7894281 thru 0x789ba91 data_size 0x1(0x7811 bytes) 2024-04-23T23:50:13.301 INFO:teuthology.orchestra.run.smithi092.stdout:1925 read 0x196831 thru 0x19ad3c (0x450c bytes) 2024-04-23T23:50:13.302 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.302 INFO:teuthology.orchestra.run.smithi092.stdout:1927 punch from 0x71a044a to 0x71a083d, (0x3f3 bytes) 2024-04-23T23:50:13.302 INFO:teuthology.orchestra.run.smithi092.stdout:1928 write 0x70d138d thru 0x70d708b (0x5cff bytes) 2024-04-23T23:50:13.303 INFO:teuthology.orchestra.run.smithi092.stdout:1931 write 0xb4f17d7 thru 0xb4fc71c (0xaf46 bytes) 2024-04-23T23:50:13.305 INFO:teuthology.orchestra.run.smithi092.stdout:1932 read 0x8f05d06 thru 0x8f0acd5 (0x4fd0 bytes) 2024-04-23T23:50:13.305 INFO:teuthology.orchestra.run.smithi092.stdout:1933 read 0x2b9ef7d thru 0x2bac33d (0xd3c1 bytes) 2024-04-23T23:50:13.305 INFO:teuthology.orchestra.run.smithi092.stdout:1934 write 0x2edbe1 thru 0x2ee0e9 (0x509 bytes) 2024-04-23T23:50:13.307 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.307 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.307 INFO:teuthology.orchestra.run.smithi092.stdout:1937 punch from 0x4592406 to 0x45a1091, (0xec8b bytes) 2024-04-23T23:50:13.307 INFO:teuthology.orchestra.run.smithi092.stdout:1938 write 0xf179eb thru 0xf2037c (0x8992 bytes) 2024-04-23T23:50:13.309 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.325 INFO:teuthology.orchestra.run.smithi092.stdout:1940 trunc from 0xe0b1fdc to 0x8a432 2024-04-23T23:50:13.344 INFO:teuthology.orchestra.run.smithi092.stdout:1943 trunc from 0x8a432 to 0x363beb6 2024-04-23T23:50:13.350 INFO:teuthology.orchestra.run.smithi092.stdout:1945 read 0x25c556e thru 0x25d1ba8 (0xc63b bytes) 2024-04-23T23:50:13.350 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.350 INFO:teuthology.orchestra.run.smithi092.stdout:1948 punch from 0x1c98304 to 0x1ca089b, (0x8597 bytes) 2024-04-23T23:50:13.355 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.355 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.364 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:13.368 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:13.382+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:13.370 INFO:teuthology.orchestra.run.smithi092.stdout:1950 write 0xea932a3 thru 0xea97616 (0x4374 bytes) 2024-04-23T23:50:13.372 INFO:teuthology.orchestra.run.smithi092.stdout:1951 write 0x537f07e thru 0x537f85a (0x7dd bytes) 2024-04-23T23:50:13.375 INFO:teuthology.orchestra.run.smithi092.stdout:1952 write 0x1938f3a thru 0x193bb99 (0x2c60 bytes) 2024-04-23T23:50:13.387 INFO:teuthology.orchestra.run.smithi092.stdout:1953 trunc from 0xea97617 to 0x5ae8640 2024-04-23T23:50:13.398 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.398 INFO:teuthology.orchestra.run.smithi092.stdout:1956 write 0x187916d thru 0x187cc14 (0x3aa8 bytes) 2024-04-23T23:50:13.414 INFO:teuthology.orchestra.run.smithi092.stdout:1957 write 0xd3a3df3 thru 0xd3b197c (0xdb8a bytes) 2024-04-23T23:50:13.418 INFO:teuthology.orchestra.run.smithi092.stdout:1958 write 0x82447df thru 0x8254110 (0xf932 bytes) 2024-04-23T23:50:13.420 INFO:teuthology.orchestra.run.smithi092.stdout:1959 read 0x58fcbd6 thru 0x58ff20c (0x2637 bytes) 2024-04-23T23:50:13.420 INFO:teuthology.orchestra.run.smithi092.stdout:1960 read 0xc048c6e thru 0xc054fa6 (0xc339 bytes) 2024-04-23T23:50:13.420 INFO:teuthology.orchestra.run.smithi092.stdout:1961 punch from 0xb607dd5 to 0xb617564, (0xf78f bytes) 2024-04-23T23:50:13.427 INFO:teuthology.orchestra.run.smithi092.stdout:1962 writesame 0xe5900e3 thru 0xe593f58 data_size 0x52(0x3e76 bytes) 2024-04-23T23:50:13.436 INFO:teuthology.orchestra.run.smithi092.stdout:1963 trunc from 0xe593f59 to 0x87c902d 2024-04-23T23:50:13.447 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.452 INFO:teuthology.orchestra.run.smithi092.stdout:1965 trunc from 0x87c902d to 0xc9d37cb 2024-04-23T23:50:13.457 INFO:teuthology.orchestra.run.smithi092.stdout:1966 read 0x5283e69 thru 0x5289931 (0x5ac9 bytes) 2024-04-23T23:50:13.462 INFO:teuthology.orchestra.run.smithi092.stdout:1968 trunc from 0xc9d37cb to 0x9178292 2024-04-23T23:50:13.469 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:13.483+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:13.474 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.474 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:13.484 INFO:teuthology.orchestra.run.smithi092.stdout:1969 write 0xecb1d94 thru 0xecbe450 (0xc6bd bytes) 2024-04-23T23:50:13.487 INFO:teuthology.orchestra.run.smithi092.stdout:1970 write 0x1c0f838 thru 0x1c1f4f5 (0xfcbe bytes) 2024-04-23T23:50:13.488 INFO:teuthology.orchestra.run.smithi092.stdout:1971 punch from 0x38c59f0 to 0x38d3881, (0xde91 bytes) 2024-04-23T23:50:13.488 INFO:teuthology.orchestra.run.smithi092.stdout:1973 write 0x66e712 thru 0x6745ab (0x5e9a bytes) 2024-04-23T23:50:13.491 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.491 INFO:teuthology.orchestra.run.smithi092.stdout:1975 write 0x624fe08 thru 0x625321e (0x3417 bytes) 2024-04-23T23:50:13.492 INFO:teuthology.orchestra.run.smithi092.stdout:1976 read 0x2b365b6 thru 0x2b428cf (0xc31a bytes) 2024-04-23T23:50:13.493 INFO:teuthology.orchestra.run.smithi092.stdout:1978 punch from 0xe548ddd to 0xe549525, (0x748 bytes) 2024-04-23T23:50:13.493 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.493 INFO:teuthology.orchestra.run.smithi092.stdout:1980 writesame 0x761f317 thru 0x762957c data_size 0xd(0xa266 bytes) 2024-04-23T23:50:13.495 INFO:teuthology.orchestra.run.smithi092.stdout:1981 read 0x382c946 thru 0x38317d8 (0x4e93 bytes) 2024-04-23T23:50:13.503 INFO:teuthology.orchestra.run.smithi092.stdout:1983 trunc from 0xecbe451 to 0x8ff248c 2024-04-23T23:50:13.514 INFO:teuthology.orchestra.run.smithi092.stdout:1986 writesame 0x7f6a51d thru 0x7f7981d data_size 0x1(0xf301 bytes) 2024-04-23T23:50:13.553 INFO:teuthology.orchestra.run.smithi092.stdout:1987 writesame 0x1072220 thru 0x107cc84 data_size 0x1(0xaa65 bytes) 2024-04-23T23:50:13.570 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:13.584+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:13.581 INFO:teuthology.orchestra.run.smithi092.stdout:1989 write 0x494dd91 thru 0x495966c (0xb8dc bytes) 2024-04-23T23:50:13.583 INFO:teuthology.orchestra.run.smithi092.stdout:1992 read 0x11d4fcf thru 0x11de8a6 (0x98d8 bytes) 2024-04-23T23:50:13.583 INFO:teuthology.orchestra.run.smithi092.stdout:1996 write 0x4c19711 thru 0x4c27aee (0xe3de bytes) 2024-04-23T23:50:13.585 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.593 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.593 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.596 INFO:teuthology.orchestra.run.smithi092.stdout:1998 write 0xd9cf3ba thru 0xd9d4d12 (0x5959 bytes) 2024-04-23T23:50:13.602 INFO:teuthology.orchestra.run.smithi092.stdout:2000 write 0xb3c9cf3 thru 0xb3cc290 (0x259e bytes) 2024-04-23T23:50:13.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:13.604 INFO:teuthology.orchestra.run.smithi092.stdout:2002 write 0xf64c4b thru 0xf675b0 (0x2966 bytes) 2024-04-23T23:50:13.604 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:13.604 INFO:teuthology.orchestra.run.smithi092.stdout:2005 punch from 0x31c9b95 to 0x31d8439, (0xe8a4 bytes) 2024-04-23T23:50:13.604 INFO:teuthology.orchestra.run.smithi092.stdout:2006 punch from 0xc1843d0 to 0xc193b8e, (0xf7be bytes) 2024-04-23T23:50:13.605 INFO:teuthology.orchestra.run.smithi092.stdout:2007 clone 13 order 22 su 131072 sc 8 2024-04-23T23:50:13.671 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:13.685+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:13.706 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.706 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:13.773 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:13.787+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:13.859 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.859 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:13.874 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:13.888+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:13.971 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:13.972 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:13.975 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:13.989+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:50:13.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:14.075 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:14.089+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:14.092 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:14.093 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:14.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:14.159 INFO:teuthology.orchestra.run.smithi092.stdout: marking collection for removal 2024-04-23T23:50:14.159 INFO:teuthology.orchestra.run.smithi092.stdout:setting '_remove' omap key 2024-04-23T23:50:14.159 INFO:teuthology.orchestra.run.smithi092.stdout:finish_remove_pgs 3.1e_head removing 3.1e 2024-04-23T23:50:14.160 INFO:teuthology.orchestra.run.smithi092.stdout:Remove successful 2024-04-23T23:50:14.176 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-04-23T23:50:14.176 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:14.190+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:14.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:14.277 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:14.290+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:14.377 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:14.391+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:14.424 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:14.424 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:14.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:14.477 INFO:teuthology.orchestra.run.smithi092.stdout:2008 write 0xebb32f thru 0xec87c8 (0xd49a bytes) 2024-04-23T23:50:14.478 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:14.492+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:14.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:14.491 INFO:teuthology.orchestra.run.smithi092.stdout:2009 writesame 0x9d6f66b thru 0x9d7772e data_size 0x86(0x80c4 bytes) 2024-04-23T23:50:14.497 INFO:teuthology.orchestra.run.smithi092.stdout:2010 write 0x9037d23 thru 0x90420e0 (0xa3be bytes) 2024-04-23T23:50:14.510 INFO:teuthology.orchestra.run.smithi092.stdout:2012 read 0x310c5cd thru 0x3119ea0 (0xd8d4 bytes) 2024-04-23T23:50:14.527 INFO:teuthology.orchestra.run.smithi092.stdout:2017 trunc from 0xd9d4d13 to 0x23db06 2024-04-23T23:50:14.553 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:14.553 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:14.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:14.574 INFO:teuthology.orchestra.run.smithi092.stdout:2018 write 0x35bf525 thru 0x35c663b (0x7117 bytes) 2024-04-23T23:50:14.579 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:14.592+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:14.579 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.579 INFO:teuthology.orchestra.run.smithi092.stdout:2020 read 0x694c43 thru 0x6a05c3 (0xb981 bytes) 2024-04-23T23:50:14.588 INFO:teuthology.orchestra.run.smithi092.stdout:2021 writesame 0x6298ddd thru 0x629df61 data_size 0x1(0x5185 bytes) 2024-04-23T23:50:14.588 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.596 INFO:teuthology.orchestra.run.smithi092.stdout:2023 trunc from 0x629df62 to 0xc63a06a 2024-04-23T23:50:14.601 INFO:teuthology.orchestra.run.smithi092.stdout:2025 write 0x3389577 thru 0x3392199 (0x8c23 bytes) 2024-04-23T23:50:14.606 INFO:teuthology.orchestra.run.smithi092.stdout:2028 write 0xabd64e2 thru 0xabe0244 (0x9d63 bytes) 2024-04-23T23:50:14.623 INFO:teuthology.orchestra.run.smithi092.stdout:2030 trunc from 0xc63a06a to 0x37303c7 2024-04-23T23:50:14.627 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:14.627 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:14.634 INFO:teuthology.orchestra.run.smithi092.stdout:2031 punch from 0x2bcc909 to 0x2bd4ba9, (0x82a0 bytes) 2024-04-23T23:50:14.637 INFO:teuthology.orchestra.run.smithi092.stdout:2035 trunc from 0x37303c7 to 0x5cb059b 2024-04-23T23:50:14.656 INFO:teuthology.orchestra.run.smithi092.stdout:2036 write 0xe55d7ac thru 0xe562805 (0x505a bytes) 2024-04-23T23:50:14.659 INFO:teuthology.orchestra.run.smithi092.stdout:2040 punch from 0xb2c7b4e to 0xb2d73ac, (0xf85e bytes) 2024-04-23T23:50:14.659 INFO:teuthology.orchestra.run.smithi092.stdout:2043 writesame 0xe254dde thru 0xe256a7a data_size 0x5(0x1c9d bytes) 2024-04-23T23:50:14.660 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:14.661 INFO:teuthology.orchestra.run.smithi092.stdout:2044 write 0x1987d9d thru 0x198eab7 (0x6d1b bytes) 2024-04-23T23:50:14.663 INFO:teuthology.orchestra.run.smithi092.stdout:2046 write 0xc328106 thru 0xc32e396 (0x6291 bytes) 2024-04-23T23:50:14.665 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.666 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.666 INFO:teuthology.orchestra.run.smithi092.stdout:2050 write 0x4a9afba thru 0x4aa10c3 (0x610a bytes) 2024-04-23T23:50:14.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1e.3 2024-04-23T23:50:14.670 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.670 INFO:teuthology.orchestra.run.smithi092.stdout:2053 punch from 0x4d33d7c to 0x4d398f8, (0x5b7c bytes) 2024-04-23T23:50:14.670 INFO:teuthology.orchestra.run.smithi092.stdout:2055 writesame 0x4460c51 thru 0x4467c6d data_size 0x9(0x701d bytes) 2024-04-23T23:50:14.677 INFO:teuthology.orchestra.run.smithi092.stdout:2057 write 0xe9756d9 thru 0xe9847cb (0xf0f3 bytes) 2024-04-23T23:50:14.680 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:14.693+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:14.680 INFO:teuthology.orchestra.run.smithi092.stdout:2058 punch from 0x5cd47cb to 0x5ce296e, (0xe1a3 bytes) 2024-04-23T23:50:14.680 INFO:teuthology.orchestra.run.smithi092.stdout:2060 punch from 0x572ee4e to 0x5731112, (0x22c4 bytes) 2024-04-23T23:50:14.681 INFO:teuthology.orchestra.run.smithi092.stdout:2061 write 0x5f5aa3c thru 0x5f5ab24 (0xe9 bytes) 2024-04-23T23:50:14.682 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.682 INFO:teuthology.orchestra.run.smithi092.stdout:2063 writesame 0x2cca34d thru 0x2cd6ff6 data_size 0x43(0xccaa bytes) 2024-04-23T23:50:14.695 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:14.695 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:2064 trunc from 0xe9847cc to 0x35e85e2 2024-04-23T23:50:14.719 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:14.719 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:14.720 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:14.720 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:14.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:566 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'3, mlcod=28'1)", 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.816380096, 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001175798, 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915528+0000", 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0580000000000001e-06 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915539+0000", 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638999999999999e-05 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915554+0000", 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5141e-05 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915565+0000", 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0789e-05 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915620+0000", 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5189999999999998e-05 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915642+0000", 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2262000000000001e-05 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916672+0000", 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010296109999999999 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916703+0000", 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1108000000000001e-05 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1147 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'4, mlcod=28'2)", 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.800295965, 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022957910000000002, 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931628+0000", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6813999999999999e-05 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931636+0000", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931644+0000", 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3359999999999997e-06 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931703+0000", 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9598000000000001e-05 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931724+0000", 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0735000000000001e-05 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933865+0000", 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002141165 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933907+0000", 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1768999999999999e-05 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1526 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'16, mlcod=29'14)", 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.346226168000001, 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021410930000000002, 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385682+0000", 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385693+0000", 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0998999999999999e-05 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385710+0000", 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7280000000000001e-05 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385719+0000", 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7940000000000008e-06 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385785+0000", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6109e-05 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385807+0000", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-05 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387778+0000", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019710769999999999 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387822+0000", 2024-04-23T23:50:14.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3665000000000003e-05 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1700 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'19, mlcod=29'17)", 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.159077076999999, 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090769799999999997, 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572830+0000", 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572832+0000", 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572856+0000", 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4743e-05 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572874+0000", 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7666e-05 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572883+0000", 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4290000000000007e-06 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572946+0000", 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000005e-05 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.572980+0000", 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4637000000000001e-05 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.573695+0000", 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071479400000000004 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.573738+0000", 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2466999999999998e-05 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1728 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'20, mlcod=29'18)", 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.116347853000001, 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022012160000000002, 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615560+0000", 2024-04-23T23:50:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.136e-06 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615580+0000", 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9409e-05 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615597+0000", 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.772e-05 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615606+0000", 2024-04-23T23:50:14.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6780000000000006e-06 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615677+0000", 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0714000000000001e-05 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615701+0000", 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4076000000000001e-05 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617717+0000", 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020162729999999999 2024-04-23T23:50:14.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617760+0000", 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3210000000000001e-05 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.272344836, 2024-04-23T23:50:14.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3876 3.9 e33/22 3:93f02760:::rbd_data.1179c62058ea.0000000000000008:head v 33'13, mlcod=33'10)", 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.264193765, 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014124350000000001, 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:14.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467714+0000", 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1100000000000004e-07 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467723+0000", 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0820000000000005e-06 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467738+0000", 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4598e-05 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467746+0000", 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0620000000000001e-06 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467806+0000", 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0591000000000003e-05 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467827+0000", 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0282e-05 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469094+0000", 2024-04-23T23:50:14.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001267481 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469126+0000", 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1427999999999998e-05 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4487 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'62, mlcod=32'60)", 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:14.751 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.067869147, 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020224290000000001, 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:14.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0599999999999999e-07 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664042+0000", 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.501e-06 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664051+0000", 2024-04-23T23:50:14.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6609999999999995e-06 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664057+0000", 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6420000000000004e-06 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664109+0000", 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664129+0000", 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0041999999999999e-05 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666021+0000", 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00189174 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666060+0000", 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9149000000000001e-05 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.755 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4629 3.0 e33/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 33'4, mlcod=31'2)", 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.052431819000001, 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002138633, 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679476+0000", 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1999999999999998e-07 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679583+0000", 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106983 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679599+0000", 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962999999999999e-05 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679608+0000", 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6659999999999997e-06 2024-04-23T23:50:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679662+0000", 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4385000000000002e-05 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679685+0000", 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2667000000000002e-05 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681580+0000", 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018954270000000001 2024-04-23T23:50:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681614+0000", 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3722e-05 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5343 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'64, mlcod=33'62)", 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.808433185, 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020278729999999999, 2024-04-23T23:50:14.760 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923480+0000", 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923490+0000", 2024-04-23T23:50:14.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3919999999999992e-06 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923497+0000", 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923545+0000", 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8647999999999999e-05 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923560+0000", 2024-04-23T23:50:14.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5065000000000001e-05 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925471+0000", 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019111340000000001 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925502+0000", 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0321e-05 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.524685875999999, 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:14.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5602 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'12, mlcod=34'10)", 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.264913+0000", 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.466994262, 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000941482, 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264913+0000", 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264913+0000", 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264914+0000", 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.251e-06 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264947+0000", 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3133999999999998e-05 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264976+0000", 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8534000000000002e-05 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.264986+0000", 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.91e-06 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265074+0000", 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8268000000000003e-05 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265099+0000", 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4828e-05 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265821+0000", 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072283999999999998 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.265854+0000", 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2716999999999999e-05 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.4 epoch 36/22 rep_tid 124 entries 36'70 (0'0) error 3:25e2e575:::rbd_data.117967c411b0.000000000000002e:head by client.4473.0:6553 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'68)", 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7128413580000004, 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027723230000000001, 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019067+0000", 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9599999999999998e-07 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019079+0000", 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2716e-05 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019107+0000", 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8232999999999999e-05 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019125+0000", 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717e-05 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019219+0000", 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3455999999999994e-05 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.021838+0000", 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026193050000000002 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 36/22 rep_tid 127 entries 36'23 (0'0) error 3:05311e05:::rbd_data.117967c411b0.0000000000000034:head by client.4473.0:6561 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'21)", 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7094520620000004, 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002186155, 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:14.777 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022456+0000", 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7899999999999998e-07 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022464+0000", 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022484+0000", 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0369000000000001e-05 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022495+0000", 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1205e-05 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022559+0000", 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3465000000000003e-05 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.024641+0000", 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020822810000000001 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6580 3.0 e36/22 3:053657a9:::rbd_data.117967c411b0.000000000000003a:head v 36'26, mlcod=36'24)", 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7005895350000007, 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011394210000000001, 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:14.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031318+0000", 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-07 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031324+0000", 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031338+0000", 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4036000000000001e-05 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031346+0000", 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2450000000000001e-06 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031392+0000", 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6214000000000002e-05 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031410+0000", 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7859e-05 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032426+0000", 2024-04-23T23:50:14.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010161530000000001 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032457+0000", 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1151e-05 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6656 3.4 e36/22 3:23b8cbcf:::rbd_data.117967c411b0.000000000000001f:head v 36'72, mlcod=36'70)", 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6833357729999996, 2024-04-23T23:50:14.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001073517, 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048572+0000", 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048579+0000", 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048593+0000", 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4331000000000001e-05 2024-04-23T23:50:14.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048601+0000", 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8539999999999997e-06 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048665+0000", 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3748e-05 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048684+0000", 2024-04-23T23:50:14.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9088000000000001e-05 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049604+0000", 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091966300000000003 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049645+0000", 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0936999999999998e-05 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.682816571, 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:14.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20374 3.0 e40/22 3:01cd6562:::rbd_data.11793b798a9c.000000000000000e:head v 40'50, mlcod=38'48)", 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.7449022830000001, 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001039091, 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987012+0000", 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9179999999999997e-06 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987022+0000", 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8320000000000001e-06 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987030+0000", 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987092+0000", 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2902000000000002e-05 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987118+0000", 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5624000000000001e-05 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.988004+0000", 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088584899999999997 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.988044+0000", 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9869000000000001e-05 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20402 3.9 e40/22 3:94675786:::rbd_data.11793b798a9c.000000000000000b:head v 40'120, mlcod=38'118)", 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:11.052683+0000", 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6792240010000001, 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000871728, 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052683+0000", 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052683+0000", 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052684+0000", 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052803+0000", 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011900099999999999 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052818+0000", 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4783999999999999e-05 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052825+0000", 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052879+0000", 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3594999999999998e-05 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.052896+0000", 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7374e-05 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.053524+0000", 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062753600000000004 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.053555+0000", 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1143999999999999e-05 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20681 3.9 e40/22 3:94675786:::rbd_data.11793b798a9c.000000000000000b:head v 40'122, mlcod=40'120)", 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:11.422297+0000", 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.309610068, 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001011762, 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422297+0000", 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422297+0000", 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422299+0000", 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.595e-06 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422367+0000", 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8240999999999999e-05 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422385+0000", 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8199999999999999e-05 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422395+0000", 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.55e-06 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422452+0000", 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7558e-05 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.422475+0000", 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3011e-05 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.423263+0000", 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078823999999999995 2024-04-23T23:50:14.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:11.423309+0000", 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5367000000000001e-05 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:14.807 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2065 writesame 0xae9df78 thru 0xaea40c0 data_size 0x1(0x6149 bytes) 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2067 write 0x8a2dcf8 thru 0x8a36a28 (0x8d31 bytes) 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2072 punch from 0x2138f97 to 0x2140e4f, (0x7eb8 bytes) 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2075 writesame 0x6da3f42 thru 0x6dac34c data_size 0xb(0x840b bytes) 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2076 read 0x573d3ec thru 0x5744805 (0x741a bytes) 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2077 read 0x60d16c1 thru 0x60df248 (0xdb88 bytes) 2024-04-23T23:50:14.809 INFO:teuthology.orchestra.run.smithi092.stdout:2078 write 0x94519c8 thru 0x945ae8e (0x94c7 bytes) 2024-04-23T23:50:14.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:14.811 INFO:teuthology.orchestra.run.smithi092.stdout:2079 trunc from 0xaea40c1 to 0x990ba5f 2024-04-23T23:50:14.811 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:2080 trunc from 0x990ba5f to 0x560ee2c 2024-04-23T23:50:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:2081 read 0x2139143 thru 0x2144ac9 (0xb987 bytes) 2024-04-23T23:50:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:2082 read 0x4e12e0a thru 0x4e21f05 (0xf0fc bytes) 2024-04-23T23:50:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:14.813 INFO:teuthology.orchestra.run.smithi092.stdout:2084 trunc from 0x560ee2c to 0xfe9b0 2024-04-23T23:50:14.813 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:14.814 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:14.814 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:14.814 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:14.815 INFO:teuthology.orchestra.run.smithi092.stdout:2085 write 0xce2eaef thru 0xce3cb1b (0xe02d bytes) 2024-04-23T23:50:14.815 INFO:teuthology.orchestra.run.smithi092.stdout:2086 read 0x2b06a74 thru 0x2b0d27e (0x680b bytes) 2024-04-23T23:50:14.815 INFO:teuthology.orchestra.run.smithi092.stdout:2087 write 0x3f4db66 thru 0x3f54550 (0x69eb bytes) 2024-04-23T23:50:14.815 INFO:teuthology.orchestra.run.smithi092.stdout:2088 punch from 0x842924b to 0x8430402, (0x71b7 bytes) 2024-04-23T23:50:14.815 INFO:teuthology.orchestra.run.smithi092.stdout:2091 write 0x1879239 thru 0x18863b5 (0xd17d bytes) 2024-04-23T23:50:14.815 INFO:teuthology.orchestra.run.smithi092.stdout:2092 writesame 0x323285b thru 0x323331e data_size 0x2(0xac4 bytes) 2024-04-23T23:50:14.816 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.816 INFO:teuthology.orchestra.run.smithi092.stdout:2095 read 0x9201596 thru 0x920bd0b (0xa776 bytes) 2024-04-23T23:50:14.816 INFO:teuthology.orchestra.run.smithi092.stdout:2100 trunc from 0xce3cb1c to 0x7d4186b 2024-04-23T23:50:14.816 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:14.822+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:14.819 INFO:teuthology.orchestra.run.smithi092.stdout:2101 punch from 0x602ddc4 to 0x602dfe6, (0x222 bytes) 2024-04-23T23:50:14.825 INFO:teuthology.orchestra.run.smithi092.stdout:2105 trunc from 0x7d4186b to 0x2d2a68d 2024-04-23T23:50:14.831 INFO:teuthology.orchestra.run.smithi092.stdout:2106 read 0x136da9b thru 0x137c534 (0xea9a bytes) 2024-04-23T23:50:14.852 INFO:teuthology.orchestra.run.smithi092.stdout:2107 writesame 0xec01f61 thru 0xec0e7f0 data_size 0x4(0xc890 bytes) 2024-04-23T23:50:14.855 INFO:teuthology.orchestra.run.smithi092.stdout:2108 trunc from 0xec0e7f1 to 0xbacc4fb 2024-04-23T23:50:14.866 INFO:teuthology.orchestra.run.smithi092.stdout:2110 write 0xc751397 thru 0xc751d5a (0x9c4 bytes) 2024-04-23T23:50:14.882 INFO:teuthology.orchestra.run.smithi092.stdout:2111 trunc from 0xc751d5b to 0x14dbf51 2024-04-23T23:50:14.908 INFO:teuthology.orchestra.run.smithi092.stdout:2113 write 0x8ddae78 thru 0x8ddc0bd (0x1246 bytes) 2024-04-23T23:50:14.909 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:14.923+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:14.911 INFO:teuthology.orchestra.run.smithi092.stdout:2115 punch from 0x8d37e3d to 0x8d3f633, (0x77f6 bytes) 2024-04-23T23:50:14.911 INFO:teuthology.orchestra.run.smithi092.stdout:2116 write 0x39fe1d9 thru 0x39ffc40 (0x1a68 bytes) 2024-04-23T23:50:14.913 INFO:teuthology.orchestra.run.smithi092.stdout:2117 read 0x86d4287 thru 0x86dcde9 (0x8b63 bytes) 2024-04-23T23:50:14.913 INFO:teuthology.orchestra.run.smithi092.stdout:2118 writesame 0xfee23e thru 0xffc20a data_size 0x1(0xdfcd bytes) 2024-04-23T23:50:14.923 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:14.923 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:14.923 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:14.923 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:14.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:14.948 INFO:teuthology.orchestra.run.smithi092.stdout:2119 writesame 0x945e238 thru 0x946c365 data_size 0x4a(0xe12e bytes) 2024-04-23T23:50:14.948 INFO:teuthology.orchestra.run.smithi092.stdout:2120 read 0x7cc3299 thru 0x7ccd3e5 (0xa14d bytes) 2024-04-23T23:50:14.949 INFO:teuthology.orchestra.run.smithi092.stdout:2124 punch from 0x8ef90de to 0x8f03fde, (0xaf00 bytes) 2024-04-23T23:50:14.958 INFO:teuthology.orchestra.run.smithi092.stdout:2125 trunc from 0x946c366 to 0x5fbd53 2024-04-23T23:50:14.974 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:14.974 INFO:teuthology.orchestra.run.smithi092.stdout:2128 read 0x47a301 thru 0x47abb8 (0x8b8 bytes) 2024-04-23T23:50:14.975 INFO:teuthology.orchestra.run.smithi092.stdout:2129 read 0x19af9c thru 0x1a2323 (0x7388 bytes) 2024-04-23T23:50:14.997 INFO:teuthology.orchestra.run.smithi092.stdout:2130 writesame 0xe208937 thru 0xe216f98 data_size 0x2(0xe662 bytes) 2024-04-23T23:50:14.997 INFO:teuthology.orchestra.run.smithi092.stdout:2131 read 0x4db5449 thru 0x4dbdf04 (0x8abc bytes) 2024-04-23T23:50:14.997 INFO:teuthology.orchestra.run.smithi092.stdout:2133 write 0x809b2d4 thru 0x80a5888 (0xa5b5 bytes) 2024-04-23T23:50:15.001 INFO:teuthology.orchestra.run.smithi092.stdout:2135 write 0x49b6b41 thru 0x49c1754 (0xac14 bytes) 2024-04-23T23:50:15.004 INFO:teuthology.orchestra.run.smithi092.stdout:2137 read 0x2ba533c thru 0x2bacb32 (0x77f7 bytes) 2024-04-23T23:50:15.012 INFO:teuthology.orchestra.run.smithi092.stdout:2138 trunc from 0xe216f99 to 0x8174bf3 2024-04-23T23:50:15.017 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:15.031+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:15.026 INFO:teuthology.orchestra.run.smithi092.stdout:2139 write 0xa3f4451 thru 0xa3fea30 (0xa5e0 bytes) 2024-04-23T23:50:15.029 INFO:teuthology.orchestra.run.smithi092.stdout:2140 writesame 0x4a0c1f3 thru 0x4a13695 data_size 0x25(0x74a3 bytes) 2024-04-23T23:50:15.033 INFO:teuthology.orchestra.run.smithi092.stdout:2142 writesame 0x603f6c7 thru 0x604ddad data_size 0x1(0xe6e7 bytes) 2024-04-23T23:50:15.071 INFO:teuthology.orchestra.run.smithi092.stdout:2143 punch from 0x469e84d to 0x46ad246, (0xe9f9 bytes) 2024-04-23T23:50:15.071 INFO:teuthology.orchestra.run.smithi092.stdout:2144 punch from 0x789609b to 0x78a387b, (0xd7e0 bytes) 2024-04-23T23:50:15.071 INFO:teuthology.orchestra.run.smithi092.stdout:2145 read 0x58f8cdf thru 0x5903014 (0xa336 bytes) 2024-04-23T23:50:15.072 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:15.073 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:15.073 INFO:teuthology.orchestra.run.smithi092.stdout:2148 write 0x228586c thru 0x228de04 (0x8599 bytes) 2024-04-23T23:50:15.076 INFO:teuthology.orchestra.run.smithi092.stdout:2149 write 0x3e36e thru 0x43ab0 (0x5743 bytes) 2024-04-23T23:50:15.078 INFO:teuthology.orchestra.run.smithi092.stdout:2150 write 0x12b1674 thru 0x12b9ea9 (0x8836 bytes) 2024-04-23T23:50:15.080 INFO:teuthology.orchestra.run.smithi092.stdout:2151 clone 14 order 20 su 32768 sc 14 2024-04-23T23:50:15.080 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:15.080 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:15.080 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:15.080 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:15.080 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:15.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:15.112 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:15.124+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:15.213 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:15.226+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:15.232 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:15.232 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:15.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:15.232 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:15.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:1143 3.14 3:2d072e84:::rbd_data.1179984961fa.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.324580307000002, 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017226057999999999, 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1143 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920896+0000", 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.920898+0000", 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.846e-06 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923748+0000", 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028504649999999999 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923773+0000", 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4311000000000002e-05 2024-04-23T23:50:15.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923784+0000", 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1212e-05 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923853+0000", 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9572000000000004e-05 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.923965+0000", 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0001114 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924349+0000", 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038400400000000002 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937275+0000", 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012926725 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938026+0000", 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075019300000000002 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938068+0000", 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2809999999999998e-05 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938122+0000", 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3520000000000003e-05 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1142 3.1b e29/22 3:dd409f93:::rbd_data.1179984961fa.0000000000000002:head v 29'17, mlcod=29'15)", 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.321412840000001, 2024-04-23T23:50:15.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014083770000000001, 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924063+0000", 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.924064+0000", 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7000000000000003e-07 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929770+0000", 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057057829999999999 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929783+0000", 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3118999999999999e-05 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929792+0000", 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6549999999999993e-06 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929854+0000", 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2498999999999999e-05 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.929871+0000", 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7007000000000002e-05 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938109+0000", 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082374490000000009 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.938147+0000", 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8287999999999997e-05 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.242 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:1148 3.a e29/22 3:575a294b:::rbd_data.1179984961fa.0000000000000007:head v 29'8, mlcod=28'6)", 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.315093998999998, 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067411720000000001, 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930382+0000", 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930383+0000", 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8999999999999995e-07 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930391+0000", 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7209999999999994e-06 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930402+0000", 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0986e-05 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930408+0000", 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.703e-06 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930466+0000", 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7485000000000003e-05 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.930485+0000", 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8936e-05 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937077+0000", 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065921360000000002 2024-04-23T23:50:15.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:57.937123+0000", 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6415000000000002e-05 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3863 3.e e33/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 33'113, mlcod=33'103)", 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.463373+0000", 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.782102968, 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078337579999999997, 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463373+0000", 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463373+0000", 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463374+0000", 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3200000000000005e-07 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463380+0000", 2024-04-23T23:50:15.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9610000000000002e-06 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463389+0000", 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5699999999999999e-06 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463397+0000", 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4549999999999998e-06 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463467+0000", 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0350000000000002e-05 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.463487+0000", 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9922999999999999e-05 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471145+0000", 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076580529999999997 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471207+0000", 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1814000000000001e-05 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:3864 3.e e33/22 3:764592b4:::rbd_object_map.1179c62058ea:head v 33'114, mlcod=33'103)", 2024-04-23T23:50:15.252 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:02.464065+0000", 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.781411092999999, 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071764840000000003, 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464065+0000", 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464065+0000", 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464066+0000", 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464071+0000", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7899999999999999e-06 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464080+0000", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6809999999999993e-06 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464085+0000", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7400000000000004e-06 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464131+0000", 2024-04-23T23:50:15.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6147999999999999e-05 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.464151+0000", 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9986000000000001e-05 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471210+0000", 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070593280000000001 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:02.471242+0000", 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1158999999999998e-05 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7323 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9299585159999992, 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065842770000000004, 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7323 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315518+0000", 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.01e-07 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315522+0000", 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315527+0000", 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.614e-06 2024-04-23T23:50:15.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315532+0000", 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5040000000000004e-06 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.321955+0000", 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006423354 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322022+0000", 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6832e-05 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322102+0000", 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0042000000000004e-05 2024-04-23T23:50:15.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7326 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315537+0000", 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9299389379999994, 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067391539999999998, 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7326 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315537+0000", 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315537+0000", 2024-04-23T23:50:15.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315538+0000", 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6199999999999999e-07 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315540+0000", 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9350000000000001e-06 2024-04-23T23:50:15.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315545+0000", 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315549+0000", 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3340000000000003e-06 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322127+0000", 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065776949999999997 2024-04-23T23:50:15.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322194+0000", 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7348000000000006e-05 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322276+0000", 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2150000000000005e-05 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7327 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9299213890000004, 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068953729999999998, 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7327 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315555+0000", 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.01e-07 2024-04-23T23:50:15.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315557+0000", 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8810000000000001e-06 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315563+0000", 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6910000000000002e-06 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315567+0000", 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5480000000000003e-06 2024-04-23T23:50:15.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322301+0000", 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067338060000000002 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322369+0000", 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7972999999999994e-05 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322450+0000", 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1073000000000001e-05 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7330 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9299036649999994, 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070575050000000004, 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7330 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315573+0000", 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9400000000000001e-07 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315575+0000", 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-04-23T23:50:15.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315580+0000", 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315584+0000", 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1030000000000004e-06 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322476+0000", 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068916070000000001 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322547+0000", 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0970999999999995e-05 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322630+0000", 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3077000000000004e-05 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7331 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9298862710000009, 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072241670000000001, 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7331 2024-04-23T23:50:15.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:15.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315590+0000", 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315592+0000", 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.796e-06 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315598+0000", 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9549999999999997e-06 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315603+0000", 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6419999999999998e-06 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322656+0000", 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007053719 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322728+0000", 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1250999999999999e-05 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322814+0000", 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6394e-05 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7333 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9298576569999994, 2024-04-23T23:50:15.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073801029999999998, 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7333 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:15.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315619+0000", 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.46e-07 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315621+0000", 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2390000000000001e-06 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315627+0000", 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9200000000000001e-06 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315632+0000", 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8860000000000003e-06 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322843+0000", 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072113359999999996 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322912+0000", 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8761000000000001e-05 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.322999+0000", 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6514999999999994e-05 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7336 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9298380930000008, 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075396389999999999, 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7336 2024-04-23T23:50:15.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315638+0000", 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.96e-07 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315640+0000", 2024-04-23T23:50:15.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7400000000000001e-06 2024-04-23T23:50:15.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315646+0000", 2024-04-23T23:50:15.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.57e-06 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315650+0000", 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3250000000000001e-06 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323025+0000", 2024-04-23T23:50:15.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007374551 2024-04-23T23:50:15.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323094+0000", 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9546000000000001e-05 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323178+0000", 2024-04-23T23:50:15.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3510999999999994e-05 2024-04-23T23:50:15.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7337 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.295 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:15.295 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9298205189999997, 2024-04-23T23:50:15.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077025030000000003, 2024-04-23T23:50:15.295 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7337 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:15.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315656+0000", 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6800000000000001e-07 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315658+0000", 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.872e-06 2024-04-23T23:50:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315664+0000", 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8479999999999997e-06 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315668+0000", 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000003e-06 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323203+0000", 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075353119999999997 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323272+0000", 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8739999999999996e-05 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323358+0000", 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6063000000000006e-05 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7340 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315673+0000", 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9298026919999991, 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078819990000000006, 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7340 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315673+0000", 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315673+0000", 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315674+0000", 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3999999999999997e-07 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315675+0000", 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.66e-06 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315681+0000", 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3789999999999997e-06 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315685+0000", 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5809999999999998e-06 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323397+0000", 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007711412 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323468+0000", 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1229000000000005e-05 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323555+0000", 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7398000000000006e-05 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7341 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9297850729999997, 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080357459999999999, 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7341 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315691+0000", 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8099999999999998e-07 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315698+0000", 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8970000000000002e-06 2024-04-23T23:50:15.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315704+0000", 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9309999999999996e-06 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315709+0000", 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5850000000000001e-06 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323583+0000", 2024-04-23T23:50:15.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007874097 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323648+0000", 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4969999999999996e-05 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323727+0000", 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8885000000000001e-05 2024-04-23T23:50:15.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7344 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9297611999999997, 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081910070000000001, 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7344 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315715+0000", 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315717+0000", 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.813e-06 2024-04-23T23:50:15.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315723+0000", 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5790000000000003e-06 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315727+0000", 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5549999999999996e-06 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323754+0000", 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080267580000000002 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323820+0000", 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5456999999999993e-05 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323906+0000", 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6446999999999994e-05 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.316 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:7345 3.14 3:2ecda676:::rbd_data.1179c4f11774.000000000000000e:a [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9297433399999999, 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083438039999999998, 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 7345 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315733+0000", 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315735+0000", 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.795e-06 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315740+0000", 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:50:15.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.315753+0000", 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2265e-05 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323934+0000", 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081817509999999993 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.323999+0000", 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4659000000000006e-05 2024-04-23T23:50:15.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:06.324077+0000", 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7509000000000006e-05 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:10306 3.1c e37/22 3:3ce1e20b:::rbd_data.1179129e5779.0000000000000004:head v 37'41, mlcod=37'39)", 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.337445+0000", 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.9080310699999998, 2024-04-23T23:50:15.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01055997, 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.337445+0000", 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.337445+0000", 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.337447+0000", 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.637e-06 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339521+0000", 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020746240000000002 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339540+0000", 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8187999999999999e-05 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339550+0000", 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0835999999999999e-05 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339616+0000", 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5654000000000005e-05 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.339634+0000", 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7995999999999998e-05 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.347943+0000", 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083094090000000002 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.348005+0000", 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1625999999999996e-05 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4473.0:15000 3.13 e37/22 3:ccff682c:::rbd_data.1179129e5779.0000000000000001:head v 37'32, mlcod=36'30)", 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.572471+0000", 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.6730047539999999, 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069512910000000001, 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.572471+0000", 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.572471+0000", 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.572472+0000", 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0079999999999999e-06 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573556+0000", 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010837259999999999 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573577+0000", 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0724e-05 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573586+0000", 2024-04-23T23:50:15.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0569999999999999e-06 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573650+0000", 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3745999999999996e-05 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.573680+0000", 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0788999999999999e-05 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579360+0000", 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056797619999999997 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579423+0000", 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2478999999999996e-05 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15281 3.14 3:28b2556a:::rbd_data.1179129e5779.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2048000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.583653+0000", 2024-04-23T23:50:15.330 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.6618236619999998, 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0098912140000000006, 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15281 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.583653+0000", 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.583653+0000", 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.583654+0000", 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8080000000000001e-06 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585797+0000", 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021424090000000001 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585813+0000", 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6249999999999999e-05 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585820+0000", 2024-04-23T23:50:15.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8329999999999996e-06 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.585886+0000", 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6019999999999995e-05 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.586025+0000", 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000138968 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.586308+0000", 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028303699999999998 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593123+0000", 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006814886 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593468+0000", 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00034489200000000002 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593513+0000", 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4876000000000002e-05 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.593544+0000", 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1235000000000002e-05 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.336 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.337 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:15.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:15.339 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:15.351+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:15.439 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:15.453+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:15.442 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:15.442 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:15.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:15.540 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:15.554+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:15.552 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:15.552 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:15.552 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:15.552 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:15.552 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:15.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:15.642 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:15.655+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14020 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 9895936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.548905+0000", 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1683525659999994, 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039402620999999999, 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14020 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548905+0000", 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548905+0000", 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548905+0000", 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7400000000000003e-07 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548907+0000", 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1320000000000001e-06 2024-04-23T23:50:15.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548915+0000", 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6990000000000003e-06 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548921+0000", 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5040000000000002e-06 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588102+0000", 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039181305999999999 2024-04-23T23:50:15.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588208+0000", 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000106308 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588307+0000", 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9098000000000006e-05 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.709 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14021 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 10059776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.548927+0000", 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1683301089999993, 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039580442, 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14021 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548927+0000", 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548927+0000", 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548984+0000", 2024-04-23T23:50:15.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6807e-05 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548987+0000", 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5349999999999999e-06 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.548995+0000", 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3380000000000005e-06 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549000+0000", 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3550000000000004e-06 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588338+0000", 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039338113000000001 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588417+0000", 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8906999999999995e-05 2024-04-23T23:50:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588508+0000", 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0387e-05 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14022 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 10485760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.714 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549008+0000", 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1682490330000004, 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039715873999999998, 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14022 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549008+0000", 2024-04-23T23:50:15.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549008+0000", 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549222+0000", 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021369499999999999 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549225+0000", 2024-04-23T23:50:15.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7709999999999998e-06 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549233+0000", 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2730000000000002e-06 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549239+0000", 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0299999999999999e-06 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588551+0000", 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039312252999999998 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588629+0000", 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7869999999999998e-05 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588724+0000", 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4982000000000006e-05 2024-04-23T23:50:15.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14023 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 10649600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549248+0000", 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1680092480000006, 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039676231999999999, 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14023 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549248+0000", 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549248+0000", 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549249+0000", 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.13e-07 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549251+0000", 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3e-06 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549258+0000", 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9410000000000001e-06 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549263+0000", 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588753+0000", 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039489602999999998 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588841+0000", 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8275000000000004e-05 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588924+0000", 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3240999999999998e-05 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.723 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14024 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549285+0000", 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1679719649999996, 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039822875000000001, 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14024 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549285+0000", 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549285+0000", 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549286+0000", 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2799999999999996e-07 2024-04-23T23:50:15.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549288+0000", 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3379999999999999e-06 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549296+0000", 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3540000000000002e-06 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549301+0000", 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.344e-06 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.588950+0000", 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039649524999999998 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589024+0000", 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627999999999994e-05 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589108+0000", 2024-04-23T23:50:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4058000000000001e-05 2024-04-23T23:50:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14025 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549310+0000", 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1679472139999998, 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039970769000000003, 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14025 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549310+0000", 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549310+0000", 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549311+0000", 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.226e-06 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549314+0000", 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2639999999999998e-06 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549320+0000", 2024-04-23T23:50:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8539999999999999e-06 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549327+0000", 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3729999999999998e-06 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589136+0000", 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039809694999999999 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589210+0000", 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3168000000000001e-05 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589281+0000", 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1188999999999999e-05 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14026 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549334+0000", 2024-04-23T23:50:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1679234049999998, 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040239100999999999, 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14026 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549334+0000", 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549334+0000", 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549334+0000", 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5700000000000002e-07 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549337+0000", 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1560000000000002e-06 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549343+0000", 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.63e-06 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549349+0000", 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589359+0000", 2024-04-23T23:50:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040010826999999999 2024-04-23T23:50:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589451+0000", 2024-04-23T23:50:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1941999999999995e-05 2024-04-23T23:50:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589573+0000", 2024-04-23T23:50:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121663 2024-04-23T23:50:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14027 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549355+0000", 2024-04-23T23:50:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1679022060000008, 2024-04-23T23:50:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040803262999999999, 2024-04-23T23:50:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14027 2024-04-23T23:50:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549355+0000", 2024-04-23T23:50:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549355+0000", 2024-04-23T23:50:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549356+0000", 2024-04-23T23:50:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6899999999999998e-07 2024-04-23T23:50:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549358+0000", 2024-04-23T23:50:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0789999999999999e-06 2024-04-23T23:50:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549364+0000", 2024-04-23T23:50:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5690000000000001e-06 2024-04-23T23:50:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549369+0000", 2024-04-23T23:50:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0000000000000004e-06 2024-04-23T23:50:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.589979+0000", 2024-04-23T23:50:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040609719000000002 2024-04-23T23:50:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.590062+0000", 2024-04-23T23:50:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2987999999999999e-05 2024-04-23T23:50:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.590158+0000", 2024-04-23T23:50:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6439000000000003e-05 2024-04-23T23:50:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14028 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549376+0000", 2024-04-23T23:50:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.167881393, 2024-04-23T23:50:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.041007741, 2024-04-23T23:50:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14028 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549376+0000", 2024-04-23T23:50:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549376+0000", 2024-04-23T23:50:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549558+0000", 2024-04-23T23:50:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018217600000000001 2024-04-23T23:50:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549561+0000", 2024-04-23T23:50:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.632e-06 2024-04-23T23:50:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549569+0000", 2024-04-23T23:50:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1710000000000002e-06 2024-04-23T23:50:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549574+0000", 2024-04-23T23:50:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4389999999999999e-06 2024-04-23T23:50:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.590250+0000", 2024-04-23T23:50:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.040675912000000002 2024-04-23T23:50:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.590311+0000", 2024-04-23T23:50:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0316000000000003e-05 2024-04-23T23:50:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.590384+0000", 2024-04-23T23:50:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3095000000000004e-05 2024-04-23T23:50:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14281 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 10993664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.566564+0000", 2024-04-23T23:50:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1506931009999999, 2024-04-23T23:50:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039489430999999998, 2024-04-23T23:50:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14281 2024-04-23T23:50:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566564+0000", 2024-04-23T23:50:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566564+0000", 2024-04-23T23:50:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566565+0000", 2024-04-23T23:50:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-07 2024-04-23T23:50:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566567+0000", 2024-04-23T23:50:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:50:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566572+0000", 2024-04-23T23:50:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:50:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566575+0000", 2024-04-23T23:50:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4020000000000002e-06 2024-04-23T23:50:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.605889+0000", 2024-04-23T23:50:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039314257999999998 2024-04-23T23:50:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.605963+0000", 2024-04-23T23:50:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4065000000000003e-05 2024-04-23T23:50:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606054+0000", 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0352000000000005e-05 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14282 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 11419648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.566581+0000", 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1506767080000007, 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039670440000000001, 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14282 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566581+0000", 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566581+0000", 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566581+0000", 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6199999999999999e-07 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566583+0000", 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9570000000000001e-06 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566589+0000", 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7869999999999998e-06 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566593+0000", 2024-04-23T23:50:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5360000000000003e-06 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606085+0000", 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039491960999999999 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606163+0000", 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7911999999999995e-05 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606251+0000", 2024-04-23T23:50:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7924999999999996e-05 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14283 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 11583488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.566599+0000", 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1506581909999998, 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039847220000000003, 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14283 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566599+0000", 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566599+0000", 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566600+0000", 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6199999999999998e-07 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566601+0000", 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.95e-06 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566607+0000", 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.558e-06 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566611+0000", 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1899999999999997e-06 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606301+0000", 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039690141999999998 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606378+0000", 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7015999999999996e-05 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606446+0000", 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7902000000000001e-05 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14284 3.5 3:a027a180:::rbd_data.117916e6274d.0000000000000000:c [read 11747328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.566618+0000", 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1506391469999997, 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039955864000000001, 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14284 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566618+0000", 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566618+0000", 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566619+0000", 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.75e-07 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566621+0000", 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0439999999999998e-06 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566627+0000", 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9560000000000002e-06 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.566631+0000", 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7149999999999997e-06 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606462+0000", 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039830800999999999 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606503+0000", 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1152999999999997e-05 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.606574+0000", 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0720000000000001e-05 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15449 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 6799360~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.645152+0000", 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0721052790000005, 2024-04-23T23:50:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039457864000000002, 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15449 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645152+0000", 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645152+0000", 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645220+0000", 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8120999999999994e-05 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645230+0000", 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8050000000000001e-06 2024-04-23T23:50:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645236+0000", 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.161e-06 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645242+0000", 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.331e-06 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684517+0000", 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039274691 2024-04-23T23:50:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684559+0000", 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2200000000000003e-05 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684610+0000", 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0555e-05 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15450 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 7225344~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.645249+0000", 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0720079630000008, 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039469874000000002, 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15450 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645249+0000", 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645249+0000", 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645250+0000", 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645251+0000", 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.53e-06 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645255+0000", 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9169999999999999e-06 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645258+0000", 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0170000000000001e-06 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684629+0000", 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039370982999999998 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684668+0000", 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8819000000000002e-05 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684719+0000", 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1184000000000003e-05 2024-04-23T23:50:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15451 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 7389184~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.645270+0000", 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0719875650000006, 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039568332999999997, 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15451 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645270+0000", 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.645270+0000", 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.646830+0000", 2024-04-23T23:50:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001560573 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.646835+0000", 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4669999999999998e-06 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.646844+0000", 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9959999999999999e-06 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.646850+0000", 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5189999999999998e-06 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684741+0000", 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.037890249000000001 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684782+0000", 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1282999999999998e-05 2024-04-23T23:50:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.684838+0000", 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6246000000000003e-05 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15466 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 10895360~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.647175+0000", 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0700824059999992, 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039449304999999997, 2024-04-23T23:50:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15466 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647175+0000", 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647175+0000", 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647175+0000", 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4200000000000002e-07 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647177+0000", 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.759e-06 2024-04-23T23:50:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647183+0000", 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5720000000000002e-06 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647187+0000", 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5789999999999997e-06 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686536+0000", 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039349238000000002 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686576+0000", 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9215999999999999e-05 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686624+0000", 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8599e-05 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15467 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 11059200~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.647193+0000", 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0700645449999993, 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039545862000000001, 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15467 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647193+0000", 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647193+0000", 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647193+0000", 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647195+0000", 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0360000000000001e-06 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647201+0000", 2024-04-23T23:50:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7370000000000003e-06 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647206+0000", 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6569999999999996e-06 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686646+0000", 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039440487000000003 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686687+0000", 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1056000000000001e-05 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686739+0000", 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1461000000000001e-05 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15468 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 11223040~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.647237+0000", 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.070020736, 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039619012000000002, 2024-04-23T23:50:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15468 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647237+0000", 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647237+0000", 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647237+0000", 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0999999999999997e-07 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647241+0000", 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9210000000000002e-06 2024-04-23T23:50:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647251+0000", 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0056e-05 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647258+0000", 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5080000000000002e-06 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686760+0000", 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039502408000000003 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686800+0000", 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9777999999999998e-05 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686856+0000", 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5430999999999998e-05 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15469 3.16 3:6d4509aa:::rbd_data.117916e6274d.0000000000000003:c [read 11386880~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.647270+0000", 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0699871309999995, 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039692058000000002, 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15469 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647270+0000", 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647270+0000", 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647271+0000", 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3099999999999998e-07 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647273+0000", 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1679999999999998e-06 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647279+0000", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7239999999999997e-06 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.647283+0000", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8210000000000001e-06 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686869+0000", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.039586058 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686912+0000", 2024-04-23T23:50:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2590999999999999e-05 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.686962+0000", 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0164999999999998e-05 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:15.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:15.824 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:15.836+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:15.824 INFO:teuthology.orchestra.run.smithi092.stdout:get_pg_num_history pg_num_history pg_num_history(e38 pg_nums {1={16=1},2={19=8},3={22=32}} deleted_pools ) 2024-04-23T23:50:15.824 INFO:teuthology.orchestra.run.smithi092.stdout:Importing pgid 3.1e 2024-04-23T23:50:15.824 INFO:teuthology.orchestra.run.smithi092.stdout:Write #3:7902102c:::rbd_object_map.117916e6274d:head# 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:snapset 0=[]:{} 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:Write #3:7a32bd15:::rbd_object_map.1179b25581a8:head# 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:snapset 0=[]:{} 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:Write #3:7c5de93e:::rbd_header.11798eaf3f6b:head# 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:snapset 0=[]:{} 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:write_pg epoch 37 info 3.1e( v 37'112 (0'0,37'112] local-lis/les=22/23 n=3 ec=22/22 lis/c=22/22 les/c/f=23/23/0 sis=22) 2024-04-23T23:50:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:Import successful 2024-04-23T23:50:15.924 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:15.938+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:15.965 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:15.966 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:15.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:15.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:50:16.030 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone13 from 0xa3fea31 (overlap 0xfe9b0) to 0x0 2024-04-23T23:50:16.060 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:16.074+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:16.083 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-04-23T23:50:16.122 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:16.122 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:16.122 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:16.122 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:16.122 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:16.125 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:16.139+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:16.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:16.226 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:16.239+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:16.276 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22324 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437280+0000", 2024-04-23T23:50:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.851298173, 2024-04-23T23:50:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018674570000000001, 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22324 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437280+0000", 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437280+0000", 2024-04-23T23:50:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437281+0000", 2024-04-23T23:50:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4499999999999998e-07 2024-04-23T23:50:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437282+0000", 2024-04-23T23:50:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.576e-06 2024-04-23T23:50:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437288+0000", 2024-04-23T23:50:16.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7019999999999998e-06 2024-04-23T23:50:16.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437292+0000", 2024-04-23T23:50:16.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5730000000000001e-06 2024-04-23T23:50:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455786+0000", 2024-04-23T23:50:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018493058 2024-04-23T23:50:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455857+0000", 2024-04-23T23:50:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1542e-05 2024-04-23T23:50:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455955+0000", 2024-04-23T23:50:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7774000000000003e-05 2024-04-23T23:50:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22325 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8512806909999999, 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018854783, 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22325 2024-04-23T23:50:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3999999999999997e-07 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437300+0000", 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5999999999999999e-06 2024-04-23T23:50:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437305+0000", 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7980000000000002e-06 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437310+0000", 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455983+0000", 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018673321 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456058+0000", 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4349000000000003e-05 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456153+0000", 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4825000000000001e-05 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22413 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437932+0000", 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8506464739999999, 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018630223000000001, 2024-04-23T23:50:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22413 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437932+0000", 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437932+0000", 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437933+0000", 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2500000000000005e-07 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437936+0000", 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9840000000000001e-06 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437955+0000", 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9893000000000001e-05 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437970+0000", 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4397e-05 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456388+0000", 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018417941 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456464+0000", 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6391999999999996e-05 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456562+0000", 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7991000000000004e-05 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22414 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437986+0000", 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8505924980000001, 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018772064000000001, 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22414 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437986+0000", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437986+0000", 2024-04-23T23:50:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437987+0000", 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9100000000000004e-07 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437989+0000", 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7999999999999999e-06 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437997+0000", 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8320000000000006e-06 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438012+0000", 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4928e-05 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456593+0000", 2024-04-23T23:50:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018581106 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456674+0000", 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1108999999999999e-05 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456758+0000", 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3697999999999997e-05 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22415 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8505574779999998, 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018925521000000001, 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22415 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0200000000000002e-07 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438024+0000", 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5280000000000002e-06 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438035+0000", 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1392e-05 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438042+0000", 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6359999999999997e-06 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456787+0000", 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018744937 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456862+0000", 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4894999999999994e-05 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456946+0000", 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4630999999999997e-05 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22416 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8505289280000001, 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019096966999999999, 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22416 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438052+0000", 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3209999999999999e-06 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438059+0000", 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6340000000000003e-06 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438064+0000", 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.587e-06 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456977+0000", 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018912192000000001 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457051+0000", 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4014e-05 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457146+0000", 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5801e-05 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22417 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8505034920000001, 2024-04-23T23:50:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019285103000000001, 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22417 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7700000000000005e-07 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438078+0000", 2024-04-23T23:50:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.255e-06 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438085+0000", 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9909999999999996e-06 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438090+0000", 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6069999999999998e-06 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457178+0000", 2024-04-23T23:50:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019087843 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457251+0000", 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2984999999999993e-05 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457360+0000", 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010894499999999999 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22418 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438097+0000", 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8504811860000001, 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019450266000000001, 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22418 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438097+0000", 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438097+0000", 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438098+0000", 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7800000000000002e-07 2024-04-23T23:50:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438100+0000", 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.007e-06 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438107+0000", 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7109999999999995e-06 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438113+0000", 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-04-23T23:50:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457391+0000", 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019277880000000001 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457459+0000", 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8281999999999994e-05 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457548+0000", 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8245999999999995e-05 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22419 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8504488370000001, 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019570586000000001, 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22419 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438139+0000", 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6950000000000006e-06 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438146+0000", 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.588e-06 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438152+0000", 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1210000000000003e-06 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457573+0000", 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019420579 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457629+0000", 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6144999999999999e-05 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457700+0000", 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0998e-05 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22420 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8504177020000001, 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019735791999999999, 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22420 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9200000000000001e-07 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438163+0000", 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.181e-06 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438170+0000", 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438175+0000", 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3029999999999999e-06 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457743+0000", 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019568312000000001 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457812+0000", 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8549999999999999e-05 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457897+0000", 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4575999999999998e-05 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22421 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8503967489999997, 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019913225, 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22421 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0499999999999999e-07 2024-04-23T23:50:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438184+0000", 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0430000000000002e-06 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438190+0000", 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1510000000000001e-06 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438201+0000", 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.066e-05 2024-04-23T23:50:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457928+0000", 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019727028000000001 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458004+0000", 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5615e-05 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458095+0000", 2024-04-23T23:50:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1323000000000006e-05 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22422 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 8716288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438208+0000", 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8503701420000001, 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020025983000000001, 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22422 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438208+0000", 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438208+0000", 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438209+0000", 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-07 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438211+0000", 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.227e-06 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438218+0000", 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7309999999999996e-06 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438228+0000", 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0553e-05 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458137+0000", 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019909194000000002 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458181+0000", 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3109999999999999e-05 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458234+0000", 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3677999999999997e-05 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22423 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438236+0000", 2024-04-23T23:50:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8503420689999999, 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020179789, 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22423 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438236+0000", 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438236+0000", 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438237+0000", 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7e-07 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438240+0000", 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4869999999999998e-06 2024-04-23T23:50:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438247+0000", 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438254+0000", 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7519999999999999e-06 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458261+0000", 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020006915 2024-04-23T23:50:16.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458326+0000", 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5377999999999996e-05 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458416+0000", 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9815999999999995e-05 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22424 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8503175220000001, 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020342560999999999, 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22424 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.343 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.46e-07 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438263+0000", 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0109999999999999e-06 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438270+0000", 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.314e-06 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438275+0000", 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.13e-06 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458443+0000", 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020168258000000001 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458514+0000", 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0993000000000003e-05 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458603+0000", 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9408999999999997e-05 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22425 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.850297844, 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020511313, 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22425 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8200000000000001e-07 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438283+0000", 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9769999999999999e-06 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438289+0000", 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3210000000000001e-06 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438295+0000", 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2329999999999997e-06 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458630+0000", 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020335189 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458701+0000", 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1175999999999998e-05 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458792+0000", 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1034999999999998e-05 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22426 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8502777999999998, 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020695265000000001, 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22426 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9999999999999998e-07 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438303+0000", 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8080000000000001e-06 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438309+0000", 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2199999999999997e-06 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438315+0000", 2024-04-23T23:50:16.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7049999999999996e-06 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458819+0000", 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020503637000000002 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458894+0000", 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5847999999999996e-05 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458996+0000", 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101547 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22427 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 13500416~487] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438322+0000", 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.850255958, 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021057026999999999, 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22427 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438322+0000", 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438322+0000", 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438323+0000", 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1199999999999998e-07 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438325+0000", 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8530000000000001e-06 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438332+0000", 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4370000000000001e-06 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438337+0000", 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459036+0000", 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020698819 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459115+0000", 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8479000000000005e-05 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459380+0000", 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026484600000000001 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22428 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438344+0000", 2024-04-23T23:50:16.360 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8502343730000002, 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021238679, 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22428 2024-04-23T23:50:16.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438344+0000", 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438344+0000", 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438345+0000", 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6899999999999998e-07 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438351+0000", 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1770000000000003e-06 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438358+0000", 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0609999999999998e-06 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438363+0000", 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3369999999999999e-06 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459413+0000", 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021049651999999999 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459491+0000", 2024-04-23T23:50:16.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8509000000000003e-05 2024-04-23T23:50:16.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459583+0000", 2024-04-23T23:50:16.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1473999999999999e-05 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22429 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.366 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438371+0000", 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.8502074840000002, 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021410074000000001, 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22429 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438371+0000", 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438371+0000", 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438372+0000", 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3900000000000005e-07 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438374+0000", 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1260000000000001e-06 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438380+0000", 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5660000000000003e-06 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438385+0000", 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7879999999999997e-06 2024-04-23T23:50:16.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459615+0000", 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021230259000000001 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459692+0000", 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7175999999999994e-05 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459781+0000", 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8620000000000002e-05 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22430 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.850183704, 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021586630999999998, 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22430 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2e-07 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438402+0000", 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.25e-06 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438409+0000", 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8109999999999997e-06 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438415+0000", 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4160000000000003e-06 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459807+0000", 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021392512999999998 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459892+0000", 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4992000000000002e-05 2024-04-23T23:50:16.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459981+0000", 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9228999999999996e-05 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.375 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:16.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /var/log/ceph/exp.3.1e.3 2024-04-23T23:50:16.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:16.378 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:16.390+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:16.379 INFO:teuthology.orchestra.run.smithi092.stdout:2152 read 0x86a12b3 thru 0x86a8c3e (0x798c bytes) 2024-04-23T23:50:16.379 INFO:teuthology.orchestra.run.smithi092.stdout:2153 write 0x193ca48 thru 0x194c215 (0xf7ce bytes) 2024-04-23T23:50:16.379 INFO:teuthology.orchestra.run.smithi092.stdout:2157 trunc from 0xa3fea31 to 0xc61ddae 2024-04-23T23:50:16.389 INFO:teuthology.orchestra.run.smithi092.stdout:2158 write 0xd8d5e81 thru 0xd8ddb71 (0x7cf1 bytes) 2024-04-23T23:50:16.393 INFO:teuthology.orchestra.run.smithi092.stdout:2159 read 0x2b2c66d thru 0x2b3123e (0x4bd2 bytes) 2024-04-23T23:50:16.393 INFO:teuthology.orchestra.run.smithi092.stdout:2160 punch from 0x5e18354 to 0x5e203d8, (0x8084 bytes) 2024-04-23T23:50:16.393 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.393 INFO:teuthology.orchestra.run.smithi092.stdout:2163 writesame 0x2b0d3c4 thru 0x2b10c1d data_size 0x1(0x385a bytes) 2024-04-23T23:50:16.403 INFO:teuthology.orchestra.run.smithi092.stdout:2164 writesame 0x34c15ea thru 0x34c6b15 data_size 0xc(0x552c bytes) 2024-04-23T23:50:16.407 INFO:teuthology.orchestra.run.smithi092.stdout:2165 read 0x504f620 thru 0x505b694 (0xc075 bytes) 2024-04-23T23:50:16.407 INFO:teuthology.orchestra.run.smithi092.stdout:2166 punch from 0xcb649c9 to 0xcb6c7a8, (0x7ddf bytes) 2024-04-23T23:50:16.407 INFO:teuthology.orchestra.run.smithi092.stdout:2167 writesame 0x94cb2b8 thru 0x94ce1d8 data_size 0x13(0x2f21 bytes) 2024-04-23T23:50:16.412 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.412 INFO:teuthology.orchestra.run.smithi092.stdout:2169 read 0x158118c thru 0x15827cd (0x1642 bytes) 2024-04-23T23:50:16.412 INFO:teuthology.orchestra.run.smithi092.stdout:2170 write 0xc00d734 thru 0xc015e56 (0x8723 bytes) 2024-04-23T23:50:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:2173 read 0x525501f thru 0x5260de6 (0xbdc8 bytes) 2024-04-23T23:50:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:2174 read 0x750e4f7 thru 0x7512253 (0x3d5d bytes) 2024-04-23T23:50:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:2175 punch from 0xd83ecb2 to 0xd844c4a, (0x5f98 bytes) 2024-04-23T23:50:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:2176 punch from 0x73c9779 to 0x73cf7eb, (0x6072 bytes) 2024-04-23T23:50:16.421 INFO:teuthology.orchestra.run.smithi092.stdout:2177 punch from 0x3b836d to 0x3bb97e, (0x3611 bytes) 2024-04-23T23:50:16.426 INFO:teuthology.orchestra.run.smithi092.stdout:2178 trunc from 0xd8ddb72 to 0x93ce4c8 2024-04-23T23:50:16.460 INFO:teuthology.orchestra.run.smithi092.stdout:2179 write 0xbb0b7ba thru 0xbb19688 (0xdecf bytes) 2024-04-23T23:50:16.466 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.466 INFO:teuthology.orchestra.run.smithi092.stdout:2181 read 0x9007a40 thru 0x900ea61 (0x7022 bytes) 2024-04-23T23:50:16.467 INFO:teuthology.orchestra.run.smithi092.stdout:2182 read 0x4f4dc1a thru 0x4f54cdb (0x70c2 bytes) 2024-04-23T23:50:16.474 INFO:teuthology.orchestra.run.smithi092.stdout:2183 trunc from 0xbb19689 to 0x5b8129e 2024-04-23T23:50:16.477 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:16.491+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:16.514 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.514 INFO:teuthology.orchestra.run.smithi092.stdout:2185 punch from 0x6d4e1f to 0x6dd237, (0x8418 bytes) 2024-04-23T23:50:16.514 INFO:teuthology.orchestra.run.smithi092.stdout:2186 write 0x4666d22 thru 0x4667d38 (0x1017 bytes) 2024-04-23T23:50:16.529 INFO:teuthology.orchestra.run.smithi092.stdout:2187 write 0xc380386 thru 0xc380ab3 (0x72e bytes) 2024-04-23T23:50:16.532 INFO:teuthology.orchestra.run.smithi092.stdout:2188 read 0x3de5db5 thru 0x3df4f68 (0xf1b4 bytes) 2024-04-23T23:50:16.532 INFO:teuthology.orchestra.run.smithi092.stdout:2190 read 0xc2245c0 thru 0xc2313c1 (0xce02 bytes) 2024-04-23T23:50:16.532 INFO:teuthology.orchestra.run.smithi092.stdout:2191 read 0xb87938f thru 0xb87f498 (0x610a bytes) 2024-04-23T23:50:16.532 INFO:teuthology.orchestra.run.smithi092.stdout:2192 write 0x9980f80 thru 0x99884d3 (0x7554 bytes) 2024-04-23T23:50:16.536 INFO:teuthology.orchestra.run.smithi092.stdout:2193 read 0x41ad412 thru 0x41b5ac7 (0x86b6 bytes) 2024-04-23T23:50:16.536 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:16.536 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:16.541 INFO:teuthology.orchestra.run.smithi092.stdout:2195 write 0xc555fdc thru 0xc564fde (0xf003 bytes) 2024-04-23T23:50:16.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:16.546 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.551 INFO:teuthology.orchestra.run.smithi092.stdout:2197 write 0xc7712e5 thru 0xc77dd4d (0xca69 bytes) 2024-04-23T23:50:16.556 INFO:teuthology.orchestra.run.smithi092.stdout:2199 writesame 0xc621bd6 thru 0xc627547 data_size 0x2(0x5972 bytes) 2024-04-23T23:50:16.562 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.569 INFO:teuthology.orchestra.run.smithi092.stdout:2201 writesame 0xe1c1472 thru 0xe1c2481 data_size 0x8(0x1010 bytes) 2024-04-23T23:50:16.569 INFO:teuthology.orchestra.run.smithi092.stdout:2203 writesame 0x23ae69b thru 0x23ba4a1 data_size 0x1(0xbe07 bytes) 2024-04-23T23:50:16.577 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:16.591+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:16.596 INFO:teuthology.orchestra.run.smithi092.stdout:2204 read 0xe03558d thru 0xe042936 (0xd3aa bytes) 2024-04-23T23:50:16.597 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.597 INFO:teuthology.orchestra.run.smithi092.stdout:2206 punch from 0x651d0fd to 0x6522633, (0x5536 bytes) 2024-04-23T23:50:16.598 INFO:teuthology.orchestra.run.smithi092.stdout:2207 trunc from 0xe1c2482 to 0xd466d40 2024-04-23T23:50:16.607 INFO:teuthology.orchestra.run.smithi092.stdout:2209 writesame 0x4fffbff thru 0x50080f5 data_size 0x1(0x84f7 bytes) 2024-04-23T23:50:16.628 INFO:teuthology.orchestra.run.smithi092.stdout:2210 write 0xa59273f thru 0xa5a13d8 (0xec9a bytes) 2024-04-23T23:50:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:2211 punch from 0x5dc2912 to 0x5dc6904, (0x3ff2 bytes) 2024-04-23T23:50:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:2213 read 0x11a1444 thru 0x11b0bff (0xf7bc bytes) 2024-04-23T23:50:16.634 INFO:teuthology.orchestra.run.smithi092.stdout:2215 write 0x257faf3 thru 0x2589b64 (0xa072 bytes) 2024-04-23T23:50:16.638 INFO:teuthology.orchestra.run.smithi092.stdout:2216 read 0x8bc7c33 thru 0x8bcc129 (0x44f7 bytes) 2024-04-23T23:50:16.638 INFO:teuthology.orchestra.run.smithi092.stdout:2218 writesame 0x4226742 thru 0x422ee0a data_size 0x5(0x86c9 bytes) 2024-04-23T23:50:16.644 INFO:teuthology.orchestra.run.smithi092.stdout:2219 read 0x4ef92f8 thru 0x4efcc43 (0x394c bytes) 2024-04-23T23:50:16.644 INFO:teuthology.orchestra.run.smithi092.stdout:2220 writesame 0x9f6ac43 thru 0x9f7943e data_size 0x93(0xe7fc bytes) 2024-04-23T23:50:16.647 INFO:teuthology.orchestra.run.smithi092.stdout:2221 read 0xa93b625 thru 0xa94754b (0xbf27 bytes) 2024-04-23T23:50:16.648 INFO:teuthology.orchestra.run.smithi092.stdout:2222 read 0xd39ce3b thru 0xd39db7a (0xd40 bytes) 2024-04-23T23:50:16.648 INFO:teuthology.orchestra.run.smithi092.stdout:2225 write 0xc33d6aa thru 0xc33e08e (0x9e5 bytes) 2024-04-23T23:50:16.650 INFO:teuthology.orchestra.run.smithi092.stdout:2226 write 0xaa4ac85 thru 0xaa523e4 (0x7760 bytes) 2024-04-23T23:50:16.655 INFO:teuthology.orchestra.run.smithi092.stdout:2227 writesame 0xa7057ce thru 0xa7081de data_size 0xb(0x2a11 bytes) 2024-04-23T23:50:16.661 INFO:teuthology.orchestra.run.smithi092.stdout:2228 read 0x726bf74 thru 0x726d7c7 (0x1854 bytes) 2024-04-23T23:50:16.661 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:16.661 INFO:teuthology.orchestra.run.smithi092.stdout:2230 clone 15 order 20 su 131072 sc 14 2024-04-23T23:50:16.679 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:16.693+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:16.693 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:16.693 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:16.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:16.780 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:16.794+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:16.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.656459216000002, 2024-04-23T23:50:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3077225479999992, 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3077060950000003, 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3076793539999993, 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3076605590000003, 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3076429750000003, 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2959474709999999, 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2957717839999994, 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2956202650000002, 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2953376389999995, 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2951917739999992, 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2942420900000009, 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2942191960000002, 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.278203821, 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2781941870000004, 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2631618259999993, 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2631405020000006, 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2631176770000003, 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2630949979999997, 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2607932749999993, 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:16.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:16.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:16.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.935 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:16.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:16.936 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:16.937 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:16.938 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:16.951+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:16.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:16.952 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:16.969 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:16.970 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:16.970 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:16.987 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:16.988 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:17.005 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:17.023 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:17.023 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:17.025 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:50:17.038 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:17.051+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:17.049 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:17.049 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:17.052 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.052 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:17.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:17.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:17.086 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone14 from 0xd466d40 (overlap 0x5b8129e) to 0x1b738c9 2024-04-23T23:50:17.138 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:17.152+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:17.210 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.210 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:17.239 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:17.253+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:17.249 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-04-23T23:50:17.251 INFO:teuthology.orchestra.run.smithi092.stdout:2232 punch from 0x4bf8985 to 0x4bfc4e0, (0x3b5b bytes) 2024-04-23T23:50:17.253 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.253 INFO:teuthology.orchestra.run.smithi092.stdout:2235 writesame 0xd28f37c thru 0xd296f8d data_size 0x1(0x7c12 bytes) 2024-04-23T23:50:17.270 INFO:teuthology.orchestra.run.smithi092.stdout:2236 writesame 0xa6bd4fb thru 0xa6cbd1c data_size 0x2b(0xe822 bytes) 2024-04-23T23:50:17.275 INFO:teuthology.orchestra.run.smithi092.stdout:2238 write 0xb85ae6 thru 0xb8cf42 (0x745d bytes) 2024-04-23T23:50:17.280 INFO:teuthology.orchestra.run.smithi092.stdout:2239 writesame 0xa6b8f2c thru 0xa6c7587 data_size 0x2e(0xe65c bytes) 2024-04-23T23:50:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:2240 punch from 0x3ba4063 to 0x3bade96, (0x9e33 bytes) 2024-04-23T23:50:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:2241 read 0x1225170 thru 0x1231cb6 (0xcb47 bytes) 2024-04-23T23:50:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:2244 read 0x6fe770c thru 0x6fef9a8 (0x829d bytes) 2024-04-23T23:50:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:2246 read 0x3d1a93e thru 0x3d1b104 (0x7c7 bytes) 2024-04-23T23:50:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:2247 write 0x36a4621 thru 0x36acd4f (0x872f bytes) 2024-04-23T23:50:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:2249 write 0xb52beb9 thru 0xb537770 (0xb8b8 bytes) 2024-04-23T23:50:17.291 INFO:teuthology.orchestra.run.smithi092.stdout:2250 punch from 0x51694fa to 0x516f475, (0x5f7b bytes) 2024-04-23T23:50:17.292 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.298 INFO:teuthology.orchestra.run.smithi092.stdout:2254 trunc from 0xd466d40 to 0x78b87fb 2024-04-23T23:50:17.326 INFO:teuthology.orchestra.run.smithi092.stdout:2255 read 0x2e42df4 thru 0x2e51c82 (0xee8f bytes) 2024-04-23T23:50:17.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.330 INFO:teuthology.orchestra.run.smithi092.stdout:2258 trunc from 0x78b87fb to 0x4f6c5a3 2024-04-23T23:50:17.340 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:17.354+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:17.363 INFO:teuthology.orchestra.run.smithi092.stdout:2259 punch from 0x49b888 to 0x49fe10, (0x4588 bytes) 2024-04-23T23:50:17.363 INFO:teuthology.orchestra.run.smithi092.stdout:2260 read 0x2e3f73d thru 0x2e4183b (0x20ff bytes) 2024-04-23T23:50:17.366 INFO:teuthology.orchestra.run.smithi092.stdout:2261 trunc from 0x4f6c5a3 to 0x26394d7 2024-04-23T23:50:17.370 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.370 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:17.409 INFO:teuthology.orchestra.run.smithi092.stdout:2262 trunc from 0x26394d7 to 0x84ede05 2024-04-23T23:50:17.413 INFO:teuthology.orchestra.run.smithi092.stdout:2263 write 0x51ac0fb thru 0x51b344f (0x7355 bytes) 2024-04-23T23:50:17.416 INFO:teuthology.orchestra.run.smithi092.stdout:2264 flatten 2024-04-23T23:50:17.441 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:17.454+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:17.473 INFO:teuthology.orchestra.run.smithi092.stdout:2265 trunc from 0x84ede05 to 0xac25ba5 2024-04-23T23:50:17.478 INFO:teuthology.orchestra.run.smithi092.stdout:2266 read 0x2d0380f thru 0x2d07e49 (0x463b bytes) 2024-04-23T23:50:17.488 INFO:teuthology.orchestra.run.smithi092.stdout:2267 write 0xeb589d5 thru 0xeb5c12a (0x3756 bytes) 2024-04-23T23:50:17.491 INFO:teuthology.orchestra.run.smithi092.stdout:2269 punch from 0xe3ff52c to 0xe401c52, (0x2726 bytes) 2024-04-23T23:50:17.491 INFO:teuthology.orchestra.run.smithi092.stdout:2271 punch from 0xc2b24ef to 0xc2bb638, (0x9149 bytes) 2024-04-23T23:50:17.502 INFO:teuthology.orchestra.run.smithi092.stdout:2273 trunc from 0xeb5c12b to 0x4e52664 2024-04-23T23:50:17.512 INFO:teuthology.orchestra.run.smithi092.stdout:2275 read 0x39bc7d7 thru 0x39ca3ec (0xdc16 bytes) 2024-04-23T23:50:17.513 INFO:teuthology.orchestra.run.smithi092.stdout:2276 read 0x27a8d6 thru 0x28915e (0xe889 bytes) 2024-04-23T23:50:17.529 INFO:teuthology.orchestra.run.smithi092.stdout:2277 write 0xe6aab82 thru 0xe6b3e66 (0x92e5 bytes) 2024-04-23T23:50:17.532 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.532 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.533 INFO:teuthology.orchestra.run.smithi092.stdout:2278 read 0x577d3a2 thru 0x5785099 (0x7cf8 bytes) 2024-04-23T23:50:17.534 INFO:teuthology.orchestra.run.smithi092.stdout:2279 read 0xd7a2f68 thru 0xd7ac41d (0x94b6 bytes) 2024-04-23T23:50:17.534 INFO:teuthology.orchestra.run.smithi092.stdout:2280 write 0x7fea9e3 thru 0x7ff1446 (0x6a64 bytes) 2024-04-23T23:50:17.536 INFO:teuthology.orchestra.run.smithi092.stdout:2281 read 0xd89fd3c thru 0xd8aba29 (0xbcee bytes) 2024-04-23T23:50:17.536 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.536 INFO:teuthology.orchestra.run.smithi092.stdout:2285 write 0x43bba25 thru 0x43caa3b (0xf017 bytes) 2024-04-23T23:50:17.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:17.542 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:17.555+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:17.542 INFO:teuthology.orchestra.run.smithi092.stdout:2286 write 0x3c4d7fa thru 0x3c4e33e (0xb45 bytes) 2024-04-23T23:50:17.545 INFO:teuthology.orchestra.run.smithi092.stdout:2287 read 0x41c1a76 thru 0x41d0d70 (0xf2fb bytes) 2024-04-23T23:50:17.545 INFO:teuthology.orchestra.run.smithi092.stdout:2288 writesame 0xd5440cf thru 0xd548ab0 data_size 0x31(0x49e2 bytes) 2024-04-23T23:50:17.547 INFO:teuthology.orchestra.run.smithi092.stdout:2290 read 0x59ba179 thru 0x59c919f (0xf027 bytes) 2024-04-23T23:50:17.548 INFO:teuthology.orchestra.run.smithi092.stdout:2291 writesame 0xb9fc720 thru 0xba0649e data_size 0x1(0x9d7f bytes) 2024-04-23T23:50:17.572 INFO:teuthology.orchestra.run.smithi092.stdout:2295 read 0x6d7c909 thru 0x6d80860 (0x3f58 bytes) 2024-04-23T23:50:17.585 INFO:teuthology.orchestra.run.smithi092.stdout:2298 trunc from 0xe6b3e67 to 0x31cd31f 2024-04-23T23:50:17.609 INFO:teuthology.orchestra.run.smithi092.stdout:2299 write 0x7ed0ed2 thru 0x7ed52de (0x440d bytes) 2024-04-23T23:50:17.611 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:2302 trunc from 0x7ed52df to 0x65c967e 2024-04-23T23:50:17.623 INFO:teuthology.orchestra.run.smithi092.stdout:2303 read 0x105d5ba thru 0x1060535 (0x2f7c bytes) 2024-04-23T23:50:17.624 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.624 INFO:teuthology.orchestra.run.smithi092.stdout:2307 punch from 0xf45828 to 0xf4ed65, (0x953d bytes) 2024-04-23T23:50:17.627 INFO:teuthology.orchestra.run.smithi092.stdout:2308 trunc from 0x65c967e to 0x38a393d 2024-04-23T23:50:17.632 INFO:teuthology.orchestra.run.smithi092.stdout:2311 read 0x8794f thru 0x919af (0xa061 bytes) 2024-04-23T23:50:17.632 INFO:teuthology.orchestra.run.smithi092.stdout:2312 writesame 0x25de2c2 thru 0x25eda30 data_size 0x1(0xf76f bytes) 2024-04-23T23:50:17.642 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:17.656+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:17.657 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.657 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:17.674 INFO:teuthology.orchestra.run.smithi092.stdout:2314 writesame 0x183e028 thru 0x1841fd2 data_size 0x3(0x3fab bytes) 2024-04-23T23:50:17.679 INFO:teuthology.orchestra.run.smithi092.stdout:2315 punch from 0xaf3ddf to 0xaff3e4, (0xb605 bytes) 2024-04-23T23:50:17.680 INFO:teuthology.orchestra.run.smithi092.stdout:2316 punch from 0x47d5d5 to 0x480c5f, (0x368a bytes) 2024-04-23T23:50:17.680 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.682 INFO:teuthology.orchestra.run.smithi092.stdout:2320 trunc from 0x38a393d to 0xe8319e 2024-04-23T23:50:17.714 INFO:teuthology.orchestra.run.smithi092.stdout:2321 write 0x61dacc1 thru 0x61e412e (0x946e bytes) 2024-04-23T23:50:17.730 INFO:teuthology.orchestra.run.smithi092.stdout:2322 write 0xcf18b96 thru 0xcf199ec (0xe57 bytes) 2024-04-23T23:50:17.733 INFO:teuthology.orchestra.run.smithi092.stdout:2323 trunc from 0xcf199ed to 0xd66e9ff 2024-04-23T23:50:17.744 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:17.758+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:17.744 INFO:teuthology.orchestra.run.smithi092.stdout:2324 trunc from 0xd66e9ff to 0x79fe02b 2024-04-23T23:50:17.753 INFO:teuthology.orchestra.run.smithi092.stdout:2325 punch from 0x5b80f0d to 0x5b81983, (0xa76 bytes) 2024-04-23T23:50:17.753 INFO:teuthology.orchestra.run.smithi092.stdout:2326 read 0x5e6136e thru 0x5e6666b (0x52fe bytes) 2024-04-23T23:50:17.753 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.758 INFO:teuthology.orchestra.run.smithi092.stdout:2330 trunc from 0x79fe02b to 0x3759aca 2024-04-23T23:50:17.774 INFO:teuthology.orchestra.run.smithi092.stdout:2331 write 0x5f43d7d thru 0x5f4c110 (0x8394 bytes) 2024-04-23T23:50:17.776 INFO:teuthology.orchestra.run.smithi092.stdout:2332 read 0x24d8cca thru 0x24df3ee (0x6725 bytes) 2024-04-23T23:50:17.787 INFO:teuthology.orchestra.run.smithi092.stdout:2333 write 0xb03ff06 thru 0xb04880d (0x8908 bytes) 2024-04-23T23:50:17.790 INFO:teuthology.orchestra.run.smithi092.stdout:2334 read 0x953a7b4 thru 0x953cbc1 (0x240e bytes) 2024-04-23T23:50:17.791 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.791 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.791 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.794 INFO:teuthology.orchestra.run.smithi092.stdout:2336 trunc from 0xb04880e to 0xe5b490f 2024-04-23T23:50:17.800 INFO:teuthology.orchestra.run.smithi092.stdout:2337 read 0x3b6915c thru 0x3b74696 (0xb53b bytes) 2024-04-23T23:50:17.800 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.801 INFO:teuthology.orchestra.run.smithi092.stdout:2339 writesame 0xafecb4 thru 0xb06d22 data_size 0x3d(0x806f bytes) 2024-04-23T23:50:17.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:17.811 INFO:teuthology.orchestra.run.smithi092.stdout:2341 trunc from 0xe5b490f to 0x80b0557 2024-04-23T23:50:17.820 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.820 INFO:teuthology.orchestra.run.smithi092.stdout:2345 writesame 0x1754943 thru 0x1764522 data_size 0xd(0xfbe0 bytes) 2024-04-23T23:50:17.825 INFO:teuthology.orchestra.run.smithi092.stdout:2346 read 0x224787c thru 0x2248ca5 (0x142a bytes) 2024-04-23T23:50:17.826 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.826 INFO:teuthology.orchestra.run.smithi092.stdout:2349 write 0xfe3e93 thru 0xfeec06 (0xad74 bytes) 2024-04-23T23:50:17.828 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.832 INFO:teuthology.orchestra.run.smithi092.stdout:2351 trunc from 0x80b0557 to 0xb3c54c3 2024-04-23T23:50:17.837 INFO:teuthology.orchestra.run.smithi092.stdout:2352 write 0x5c09db6 thru 0x5c0e32a (0x4575 bytes) 2024-04-23T23:50:17.840 INFO:teuthology.orchestra.run.smithi092.stdout:2353 writesame 0xa1e5edd thru 0xa1e97fa data_size 0x3(0x391e bytes) 2024-04-23T23:50:17.845 INFO:teuthology.orchestra.run.smithi092.stdout:2354 read 0x597c6c0 thru 0x598a1c4 (0xdb05 bytes) 2024-04-23T23:50:17.845 INFO:teuthology.orchestra.run.smithi092.stdout:2356 read 0xb1385c2 thru 0xb146afe (0xe53d bytes) 2024-04-23T23:50:17.845 INFO:teuthology.orchestra.run.smithi092.stdout:2357 read 0x2514a83 thru 0x251bb39 (0x70b7 bytes) 2024-04-23T23:50:17.845 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.845 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:17.858+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:17.858 INFO:teuthology.orchestra.run.smithi092.stdout:2361 trunc from 0xb3c54c3 to 0x720b40 2024-04-23T23:50:17.882 INFO:teuthology.orchestra.run.smithi092.stdout:2362 write 0x640be87 thru 0x6419129 (0xd2a3 bytes) 2024-04-23T23:50:17.885 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.885 INFO:teuthology.orchestra.run.smithi092.stdout:2364 read 0x5587584 thru 0x559228d (0xad0a bytes) 2024-04-23T23:50:17.887 INFO:teuthology.orchestra.run.smithi092.stdout:2366 trunc from 0x641912a to 0x7994179 2024-04-23T23:50:17.892 INFO:teuthology.orchestra.run.smithi092.stdout:2367 read 0x5527840 thru 0x5530a15 (0x91d6 bytes) 2024-04-23T23:50:17.892 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.892 INFO:teuthology.orchestra.run.smithi092.stdout:2370 writesame 0x2f968ef thru 0x2f9bcfb data_size 0x1(0x540d bytes) 2024-04-23T23:50:17.917 INFO:teuthology.orchestra.run.smithi092.stdout:2371 write 0xc0e006d thru 0xc0edcc7 (0xdc5b bytes) 2024-04-23T23:50:17.920 INFO:teuthology.orchestra.run.smithi092.stdout:2372 read 0x7e82f28 thru 0x7e83f4f (0x1028 bytes) 2024-04-23T23:50:17.928 INFO:teuthology.orchestra.run.smithi092.stdout:2374 trunc from 0xc0edcc8 to 0x59040f5 2024-04-23T23:50:17.929 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:17.929 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:17.937 INFO:teuthology.orchestra.run.smithi092.stdout:2375 read 0x3c40c28 thru 0x3c477df (0x6bb8 bytes) 2024-04-23T23:50:17.937 INFO:teuthology.orchestra.run.smithi092.stdout:2376 read 0x5c5c3b thru 0x5c9621 (0x39e7 bytes) 2024-04-23T23:50:17.937 INFO:teuthology.orchestra.run.smithi092.stdout:2378 writesame 0x42f9eb4 thru 0x4309039 data_size 0x12(0xf186 bytes) 2024-04-23T23:50:17.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:17.942 INFO:teuthology.orchestra.run.smithi092.stdout:2379 read 0x31370bd thru 0x3138bf3 (0x1b37 bytes) 2024-04-23T23:50:17.942 INFO:teuthology.orchestra.run.smithi092.stdout:2380 write 0x3e9671f thru 0x3ea17b7 (0xb099 bytes) 2024-04-23T23:50:17.945 INFO:teuthology.orchestra.run.smithi092.stdout:2381 punch from 0x3c4b344 to 0x3c5461d, (0x92d9 bytes) 2024-04-23T23:50:17.945 INFO:teuthology.orchestra.run.smithi092.stdout:2382 punch from 0x24a53c8 to 0x24b3247, (0xde7f bytes) 2024-04-23T23:50:17.946 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:17.959+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:17.959 INFO:teuthology.orchestra.run.smithi092.stdout:2383 writesame 0xca6f48c thru 0xca70197 data_size 0xa(0xd0c bytes) 2024-04-23T23:50:17.959 INFO:teuthology.orchestra.run.smithi092.stdout:2384 punch from 0x2b642f8 to 0x2b724cf, (0xe1d7 bytes) 2024-04-23T23:50:17.959 INFO:teuthology.orchestra.run.smithi092.stdout:2385 read 0x56b6210 thru 0x56b9a88 (0x3879 bytes) 2024-04-23T23:50:17.969 INFO:teuthology.orchestra.run.smithi092.stdout:2386 trunc from 0xca70198 to 0x3aae538 2024-04-23T23:50:17.975 INFO:teuthology.orchestra.run.smithi092.stdout:2387 read 0x235a110 thru 0x236856f (0xe460 bytes) 2024-04-23T23:50:17.975 INFO:teuthology.orchestra.run.smithi092.stdout:2388 punch from 0x2b98200 to 0x2b9894a, (0x74a bytes) 2024-04-23T23:50:17.975 INFO:teuthology.orchestra.run.smithi092.stdout:2390 punch from 0x306591d to 0x30709e2, (0xb0c5 bytes) 2024-04-23T23:50:17.989 INFO:teuthology.orchestra.run.smithi092.stdout:2391 write 0xb1f4d9c thru 0xb1f8f97 (0x41fc bytes) 2024-04-23T23:50:17.992 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:17.992 INFO:teuthology.orchestra.run.smithi092.stdout:2393 write 0x6650298 thru 0x6654ba3 (0x490c bytes) 2024-04-23T23:50:17.994 INFO:teuthology.orchestra.run.smithi092.stdout:2394 writesame 0x7d10626 thru 0x7d12f5b data_size 0x5(0x2936 bytes) 2024-04-23T23:50:17.998 INFO:teuthology.orchestra.run.smithi092.stdout:2395 read 0x5be90d0 thru 0x5bf4df8 (0xbd29 bytes) 2024-04-23T23:50:17.998 INFO:teuthology.orchestra.run.smithi092.stdout:2397 write 0x7593d6f thru 0x7599127 (0x53b9 bytes) 2024-04-23T23:50:18.000 INFO:teuthology.orchestra.run.smithi092.stdout:2398 punch from 0x2370982 to 0x2375eeb, (0x5569 bytes) 2024-04-23T23:50:18.000 INFO:teuthology.orchestra.run.smithi092.stdout:2400 write 0x78c34b1 thru 0x78d1335 (0xde85 bytes) 2024-04-23T23:50:18.003 INFO:teuthology.orchestra.run.smithi092.stdout:2402 punch from 0x8154e44 to 0x815e562, (0x971e bytes) 2024-04-23T23:50:18.010 INFO:teuthology.orchestra.run.smithi092.stdout:2404 write 0xce87033 thru 0xce8a499 (0x3467 bytes) 2024-04-23T23:50:18.012 INFO:teuthology.orchestra.run.smithi092.stdout:2405 writesame 0x4672216 thru 0x4677f2d data_size 0x24(0x5d18 bytes) 2024-04-23T23:50:18.015 INFO:teuthology.orchestra.run.smithi092.stdout:2406 read 0x3f0b698 thru 0x3f1a6e4 (0xf04d bytes) 2024-04-23T23:50:18.015 INFO:teuthology.orchestra.run.smithi092.stdout:2407 writesame 0x3f3ac26 thru 0x3f4a581 data_size 0x2(0xf95c bytes) 2024-04-23T23:50:18.034 INFO:teuthology.orchestra.run.smithi092.stdout:2408 write 0x2ad77db thru 0x2adae3d (0x3663 bytes) 2024-04-23T23:50:18.037 INFO:teuthology.orchestra.run.smithi092.stdout:2409 punch from 0xb65e4b4 to 0xb66d68e, (0xf1da bytes) 2024-04-23T23:50:18.037 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.037 INFO:teuthology.orchestra.run.smithi092.stdout:2411 trunc from 0xce8a49a to 0xd17925c 2024-04-23T23:50:18.041 INFO:teuthology.orchestra.run.smithi092.stdout:2412 writesame 0x24785a thru 0x2492cb data_size 0x2(0x1a72 bytes) 2024-04-23T23:50:18.046 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:18.059+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:18.059 INFO:teuthology.orchestra.run.smithi092.stdout:2415 trunc from 0xd17925c to 0x11c3df0 2024-04-23T23:50:18.061 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.061 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:18.074 INFO:teuthology.orchestra.run.smithi092.stdout:2416 trunc from 0x11c3df0 to 0x26aaa60 2024-04-23T23:50:18.090 INFO:teuthology.orchestra.run.smithi092.stdout:2417 writesame 0x81c5bc2 thru 0x81c9701 data_size 0xc(0x3b40 bytes) 2024-04-23T23:50:18.091 INFO:teuthology.orchestra.run.smithi092.stdout:2419 read 0x264507a thru 0x264d51f (0x84a6 bytes) 2024-04-23T23:50:18.091 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.097 INFO:teuthology.orchestra.run.smithi092.stdout:2423 write 0x966aa33 thru 0x9675721 (0xacef bytes) 2024-04-23T23:50:18.100 INFO:teuthology.orchestra.run.smithi092.stdout:2424 read 0x860424b thru 0x860e3ef (0xa1a5 bytes) 2024-04-23T23:50:18.100 INFO:teuthology.orchestra.run.smithi092.stdout:2425 read 0x429f275 thru 0x42a05a5 (0x1331 bytes) 2024-04-23T23:50:18.100 INFO:teuthology.orchestra.run.smithi092.stdout:2426 punch from 0x7059f50 to 0x7060009, (0x60b9 bytes) 2024-04-23T23:50:18.100 INFO:teuthology.orchestra.run.smithi092.stdout:2428 write 0x5913b3d thru 0x59217b3 (0xdc77 bytes) 2024-04-23T23:50:18.103 INFO:teuthology.orchestra.run.smithi092.stdout:2429 punch from 0x6e97eab to 0x6e9acac, (0x2e01 bytes) 2024-04-23T23:50:18.108 INFO:teuthology.orchestra.run.smithi092.stdout:2433 write 0x9c1e214 thru 0x9c1f7d6 (0x15c3 bytes) 2024-04-23T23:50:18.111 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.111 INFO:teuthology.orchestra.run.smithi092.stdout:2436 write 0x5d0fd79 thru 0x5d119fa (0x1c82 bytes) 2024-04-23T23:50:18.120 INFO:teuthology.orchestra.run.smithi092.stdout:2437 writesame 0xc01e46d thru 0xc01fab8 data_size 0x2(0x164c bytes) 2024-04-23T23:50:18.127 INFO:teuthology.orchestra.run.smithi092.stdout:2438 trunc from 0xc01fab9 to 0x659e118 2024-04-23T23:50:18.136 INFO:teuthology.orchestra.run.smithi092.stdout:2439 read 0x45e8abd thru 0x45ef5df (0x6b23 bytes) 2024-04-23T23:50:18.136 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.136 INFO:teuthology.orchestra.run.smithi092.stdout:2441 trunc from 0x659e118 to 0x596d914 2024-04-23T23:50:18.147 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:18.161+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:2443 write 0xe4bccb6 thru 0xe4c1a54 (0x4d9f bytes) 2024-04-23T23:50:18.165 INFO:teuthology.orchestra.run.smithi092.stdout:2444 trunc from 0xe4c1a55 to 0xc0fcb7c 2024-04-23T23:50:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:2446 punch from 0x50f0982 to 0x50f0bbf, (0x23d bytes) 2024-04-23T23:50:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:2447 read 0xa26ae25 thru 0xa270a8d (0x5c69 bytes) 2024-04-23T23:50:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.182 INFO:teuthology.orchestra.run.smithi092.stdout:2452 write 0xdd4ae32 thru 0xdd4dda8 (0x2f77 bytes) 2024-04-23T23:50:18.190 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.190 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.193 INFO:teuthology.orchestra.run.smithi092.stdout:2454 trunc from 0xdd4dda9 to 0x71a59c8 2024-04-23T23:50:18.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:18.203 INFO:teuthology.orchestra.run.smithi092.stdout:2455 read 0x4576c7a thru 0x4577aaa (0xe31 bytes) 2024-04-23T23:50:18.203 INFO:teuthology.orchestra.run.smithi092.stdout:2458 read 0x3b29d2f thru 0x3b2d741 (0x3a13 bytes) 2024-04-23T23:50:18.203 INFO:teuthology.orchestra.run.smithi092.stdout:2459 read 0x171ef6c thru 0x1725093 (0x6128 bytes) 2024-04-23T23:50:18.203 INFO:teuthology.orchestra.run.smithi092.stdout:2460 write 0x2021c8b thru 0x202d4f3 (0xb869 bytes) 2024-04-23T23:50:18.206 INFO:teuthology.orchestra.run.smithi092.stdout:2462 trunc from 0x71a59c8 to 0x6c73d53 2024-04-23T23:50:18.217 INFO:teuthology.orchestra.run.smithi092.stdout:2463 trunc from 0x6c73d53 to 0x1c7adf8 2024-04-23T23:50:18.229 INFO:teuthology.orchestra.run.smithi092.stdout:2464 read 0x17d1e6a thru 0x17d5e74 (0x400b bytes) 2024-04-23T23:50:18.229 INFO:teuthology.orchestra.run.smithi092.stdout:2467 read 0x13199ca thru 0x131d9a6 (0x3fdd bytes) 2024-04-23T23:50:18.235 INFO:teuthology.orchestra.run.smithi092.stdout:2471 write 0x2769fb0 thru 0x27714c0 (0x7511 bytes) 2024-04-23T23:50:18.248 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:18.262+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:18.255 INFO:teuthology.orchestra.run.smithi092.stdout:2472 write 0xd5f7e91 thru 0xd5f801c (0x18c bytes) 2024-04-23T23:50:18.258 INFO:teuthology.orchestra.run.smithi092.stdout:2474 punch from 0x9b190ea to 0x9b21dfd, (0x8d13 bytes) 2024-04-23T23:50:18.258 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.258 INFO:teuthology.orchestra.run.smithi092.stdout:2476 write 0x4b6cfa6 thru 0x4b72605 (0x5660 bytes) 2024-04-23T23:50:18.260 INFO:teuthology.orchestra.run.smithi092.stdout:2477 read 0x16472d8 thru 0x1651d45 (0xaa6e bytes) 2024-04-23T23:50:18.260 INFO:teuthology.orchestra.run.smithi092.stdout:2479 writesame 0x78518de thru 0x7851a0f data_size 0x3(0x132 bytes) 2024-04-23T23:50:18.263 INFO:teuthology.orchestra.run.smithi092.stdout:2480 read 0x2c9d13f thru 0x2c9db70 (0xa32 bytes) 2024-04-23T23:50:18.263 INFO:teuthology.orchestra.run.smithi092.stdout:2481 punch from 0x81e844b to 0x81f52d2, (0xce87 bytes) 2024-04-23T23:50:18.263 INFO:teuthology.orchestra.run.smithi092.stdout:2482 read 0x8eaaee9 thru 0x8eba14f (0xf267 bytes) 2024-04-23T23:50:18.263 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:18.263 INFO:teuthology.orchestra.run.smithi092.stdout:2484 clone 16 order 25 su 262144 sc 11 2024-04-23T23:50:18.310 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.310 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.318 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:18.349 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:18.363+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:18.450 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:18.464+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:18.462 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.462 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.471 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:18.556 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:18.570+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:18.613 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.613 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:18.652 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:18.666+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:18.753 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:18.767+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:18.763 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.764 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:18.857 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:18.871+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:18.915 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:18.915 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:18.923 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:18.955 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:18.969+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:19.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:50:19.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:19.070 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:19.071 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:19.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:19.090 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:19.104+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:19.156 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:19.170+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:19.184 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:19.185 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:19.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:19.231 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:19.231 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:19.249 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-04-23T23:50:19.250 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:19.251 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:19.275 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:19.275 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:19.275 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:19.277 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:19.291+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:19.295 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:19.316 INFO:teuthology.orchestra.run.smithi092.stdout:2485 writesame 0x4138911 thru 0x414650c data_size 0x9c(0xdbfc bytes) 2024-04-23T23:50:19.322 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:19.326 INFO:teuthology.orchestra.run.smithi092.stdout:2486 writesame 0xcf7f05 thru 0xcf8037 data_size 0x1(0x133 bytes) 2024-04-23T23:50:19.327 INFO:teuthology.orchestra.run.smithi092.stdout:2487 read 0x23fc9fa thru 0x2403c12 (0x7219 bytes) 2024-04-23T23:50:19.328 INFO:teuthology.orchestra.run.smithi092.stdout:2488 writesame 0x34df152 thru 0x34e7477 data_size 0x1(0x8326 bytes) 2024-04-23T23:50:19.346 INFO:teuthology.orchestra.run.smithi092.stdout:2490 read 0xc97d38e thru 0xc98303a (0x5cad bytes) 2024-04-23T23:50:19.348 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:19.348 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:19.348 INFO:teuthology.orchestra.run.smithi092.stdout:2491 write 0x9b05ce7 thru 0x9b0a532 (0x484c bytes) 2024-04-23T23:50:19.353 INFO:teuthology.orchestra.run.smithi092.stdout:2492 write 0x2c109ba thru 0x2c12afa (0x2141 bytes) 2024-04-23T23:50:19.354 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:19.354 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:19.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:19.359 INFO:teuthology.orchestra.run.smithi092.stdout:2493 read 0x91b9543 thru 0x91c0f65 (0x7a23 bytes) 2024-04-23T23:50:19.359 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.359 INFO:teuthology.orchestra.run.smithi092.stdout:2495 read 0xbe78df6 thru 0xbe872e4 (0xe4ef bytes) 2024-04-23T23:50:19.370 INFO:teuthology.orchestra.run.smithi092.stdout:2496 trunc from 0xd5f801d to 0x4994a1f 2024-04-23T23:50:19.381 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:19.381 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:19.384 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:19.385 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:19.400 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:19.413+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:19.414 INFO:teuthology.orchestra.run.smithi092.stdout:2499 write 0xdccef81 thru 0xdccf766 (0x7e6 bytes) 2024-04-23T23:50:19.415 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.415 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.415 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.415 INFO:teuthology.orchestra.run.smithi092.stdout:2504 writesame 0x6dc0ffb thru 0x6dcf79b data_size 0x2b(0xe7a1 bytes) 2024-04-23T23:50:19.417 INFO:teuthology.orchestra.run.smithi092.stdout:2505 read 0x64ccfb4 thru 0x64cf1da (0x2227 bytes) 2024-04-23T23:50:19.424 INFO:teuthology.orchestra.run.smithi092.stdout:2507 trunc from 0xdccf767 to 0x846bbd9 2024-04-23T23:50:19.438 INFO:teuthology.orchestra.run.smithi092.stdout:2508 read 0x4dd31b5 thru 0x4dd6d13 (0x3b5f bytes) 2024-04-23T23:50:19.439 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.439 INFO:teuthology.orchestra.run.smithi092.stdout:2510 writesame 0x11b9b46 thru 0x11c852d data_size 0x5(0xe9e8 bytes) 2024-04-23T23:50:19.448 INFO:teuthology.orchestra.run.smithi092.stdout:2511 write 0x2e16d41 thru 0x2e1df97 (0x7257 bytes) 2024-04-23T23:50:19.454 INFO:teuthology.orchestra.run.smithi092.stdout:2512 trunc from 0x846bbd9 to 0x476ac47 2024-04-23T23:50:19.460 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:19.473+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:19.466 INFO:teuthology.orchestra.run.smithi092.stdout:2513 read 0x1807187 thru 0x1811c07 (0xaa81 bytes) 2024-04-23T23:50:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:2514 writesame 0x84d8841 thru 0x84e13d7 data_size 0x7(0x8b97 bytes) 2024-04-23T23:50:19.488 INFO:teuthology.orchestra.run.smithi092.stdout:2515 write 0xbe0bbfe thru 0xbe0fe1c (0x421f bytes) 2024-04-23T23:50:19.489 INFO:teuthology.orchestra.run.smithi092.stdout:2518 write 0x24d158c thru 0x24d3de3 (0x2858 bytes) 2024-04-23T23:50:19.491 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.498 INFO:teuthology.orchestra.run.smithi092.stdout:2522 trunc from 0xbe0fe1d to 0x5f195ba 2024-04-23T23:50:19.511 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22324 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437280+0000", 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0873333330000001, 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018674570000000001, 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22324 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437280+0000", 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437280+0000", 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437281+0000", 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4499999999999998e-07 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437282+0000", 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.576e-06 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437288+0000", 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7019999999999998e-06 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437292+0000", 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5730000000000001e-06 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455786+0000", 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018493058 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455857+0000", 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1542e-05 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455955+0000", 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7774000000000003e-05 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22325 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0873158509999996, 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018854783, 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22325 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437298+0000", 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3999999999999997e-07 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437300+0000", 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5999999999999999e-06 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437305+0000", 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7980000000000002e-06 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437310+0000", 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.455983+0000", 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018673321 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456058+0000", 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4349000000000003e-05 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456153+0000", 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4825000000000001e-05 2024-04-23T23:50:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22413 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437932+0000", 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0866816339999996, 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018630223000000001, 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22413 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437932+0000", 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437932+0000", 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437933+0000", 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2500000000000005e-07 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437936+0000", 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9840000000000001e-06 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437955+0000", 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9893000000000001e-05 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437970+0000", 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4397e-05 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456388+0000", 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018417941 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456464+0000", 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6391999999999996e-05 2024-04-23T23:50:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456562+0000", 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7991000000000004e-05 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22414 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.437986+0000", 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0866276580000003, 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018772064000000001, 2024-04-23T23:50:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22414 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437986+0000", 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437986+0000", 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437987+0000", 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9100000000000004e-07 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437989+0000", 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7999999999999999e-06 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.437997+0000", 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8320000000000006e-06 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438012+0000", 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4928e-05 2024-04-23T23:50:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456593+0000", 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018581106 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456674+0000", 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1108999999999999e-05 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456758+0000", 2024-04-23T23:50:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3697999999999997e-05 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22415 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0865926379999999, 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018925521000000001, 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22415 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438021+0000", 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0200000000000002e-07 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438024+0000", 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5280000000000002e-06 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438035+0000", 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1392e-05 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438042+0000", 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6359999999999997e-06 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456787+0000", 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018744937 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456862+0000", 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4894999999999994e-05 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456946+0000", 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4630999999999997e-05 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22416 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0865640880000003, 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019096966999999999, 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22416 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438050+0000", 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438052+0000", 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3209999999999999e-06 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438059+0000", 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6340000000000003e-06 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438064+0000", 2024-04-23T23:50:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.587e-06 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.456977+0000", 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018912192000000001 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457051+0000", 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4014e-05 2024-04-23T23:50:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457146+0000", 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5801e-05 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22417 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0865386519999998, 2024-04-23T23:50:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019285103000000001, 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22417 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438075+0000", 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7700000000000005e-07 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438078+0000", 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.255e-06 2024-04-23T23:50:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438085+0000", 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9909999999999996e-06 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438090+0000", 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6069999999999998e-06 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457178+0000", 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019087843 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457251+0000", 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2984999999999993e-05 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457360+0000", 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010894499999999999 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22418 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438097+0000", 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0865163459999998, 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019450266000000001, 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22418 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438097+0000", 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438097+0000", 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438098+0000", 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7800000000000002e-07 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438100+0000", 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.007e-06 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438107+0000", 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7109999999999995e-06 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438113+0000", 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457391+0000", 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019277880000000001 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457459+0000", 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8281999999999994e-05 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457548+0000", 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8245999999999995e-05 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22419 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0864839970000002, 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019570586000000001, 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22419 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438130+0000", 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438139+0000", 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6950000000000006e-06 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438146+0000", 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.588e-06 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438152+0000", 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1210000000000003e-06 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457573+0000", 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019420579 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457629+0000", 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6144999999999999e-05 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457700+0000", 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0998e-05 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22420 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0864528619999998, 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019735791999999999, 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22420 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438161+0000", 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9200000000000001e-07 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438163+0000", 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.181e-06 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438170+0000", 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438175+0000", 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3029999999999999e-06 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457743+0000", 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019568312000000001 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457812+0000", 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8549999999999999e-05 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457897+0000", 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4575999999999998e-05 2024-04-23T23:50:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22421 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0864319089999999, 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019913225, 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22421 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438182+0000", 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0499999999999999e-07 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438184+0000", 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0430000000000002e-06 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438190+0000", 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1510000000000001e-06 2024-04-23T23:50:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438201+0000", 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.066e-05 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.457928+0000", 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019727028000000001 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458004+0000", 2024-04-23T23:50:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5615e-05 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458095+0000", 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1323000000000006e-05 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22422 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 8716288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438208+0000", 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0864053020000002, 2024-04-23T23:50:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020025983000000001, 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22422 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438208+0000", 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438208+0000", 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438209+0000", 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-07 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438211+0000", 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.227e-06 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438218+0000", 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7309999999999996e-06 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438228+0000", 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0553e-05 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458137+0000", 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019909194000000002 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458181+0000", 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3109999999999999e-05 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458234+0000", 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3677999999999997e-05 2024-04-23T23:50:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22423 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438236+0000", 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.086377229, 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020179789, 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22423 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438236+0000", 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438236+0000", 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438237+0000", 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7e-07 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438240+0000", 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4869999999999998e-06 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438247+0000", 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438254+0000", 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7519999999999999e-06 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458261+0000", 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020006915 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458326+0000", 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5377999999999996e-05 2024-04-23T23:50:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458416+0000", 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9815999999999995e-05 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22424 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0863526820000002, 2024-04-23T23:50:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020342560999999999, 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22424 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438261+0000", 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.46e-07 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438263+0000", 2024-04-23T23:50:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0109999999999999e-06 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438270+0000", 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.314e-06 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438275+0000", 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.13e-06 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458443+0000", 2024-04-23T23:50:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020168258000000001 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458514+0000", 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0993000000000003e-05 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458603+0000", 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9408999999999997e-05 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22425 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0863330040000001, 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020511313, 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22425 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438281+0000", 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8200000000000001e-07 2024-04-23T23:50:19.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438283+0000", 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9769999999999999e-06 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438289+0000", 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3210000000000001e-06 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438295+0000", 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2329999999999997e-06 2024-04-23T23:50:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458630+0000", 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020335189 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458701+0000", 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1175999999999998e-05 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458792+0000", 2024-04-23T23:50:19.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1034999999999998e-05 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22426 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0863129599999999, 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020695265000000001, 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22426 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438301+0000", 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9999999999999998e-07 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438303+0000", 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8080000000000001e-06 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438309+0000", 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2199999999999997e-06 2024-04-23T23:50:19.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438315+0000", 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7049999999999996e-06 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458819+0000", 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020503637000000002 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458894+0000", 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5847999999999996e-05 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.458996+0000", 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101547 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22427 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 13500416~487] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.583 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438322+0000", 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0862911180000001, 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021057026999999999, 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22427 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438322+0000", 2024-04-23T23:50:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438322+0000", 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438323+0000", 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1199999999999998e-07 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438325+0000", 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8530000000000001e-06 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438332+0000", 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4370000000000001e-06 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438337+0000", 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-04-23T23:50:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459036+0000", 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.020698819 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459115+0000", 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8479000000000005e-05 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459380+0000", 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026484600000000001 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22428 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438344+0000", 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0862695330000003, 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021238679, 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.588 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22428 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438344+0000", 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438344+0000", 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438345+0000", 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6899999999999998e-07 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438351+0000", 2024-04-23T23:50:19.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1770000000000003e-06 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438358+0000", 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0609999999999998e-06 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438363+0000", 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3369999999999999e-06 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459413+0000", 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021049651999999999 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459491+0000", 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8509000000000003e-05 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459583+0000", 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1473999999999999e-05 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.592 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22429 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438371+0000", 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0862426440000004, 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021410074000000001, 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22429 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438371+0000", 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438371+0000", 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438372+0000", 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3900000000000005e-07 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438374+0000", 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1260000000000001e-06 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438380+0000", 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5660000000000003e-06 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438385+0000", 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7879999999999997e-06 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459615+0000", 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021230259000000001 2024-04-23T23:50:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459692+0000", 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7175999999999994e-05 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459781+0000", 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8620000000000002e-05 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:22430 3.a 3:57737f67:::rbd_data.11793b798a9c.000000000000000a:e [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.0862188640000001, 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021586630999999998, 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.598 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 22430 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438395+0000", 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2e-07 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438402+0000", 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.25e-06 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438409+0000", 2024-04-23T23:50:19.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8109999999999997e-06 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.438415+0000", 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4160000000000003e-06 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459807+0000", 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021392512999999998 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459892+0000", 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4992000000000002e-05 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:12.459981+0000", 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9228999999999996e-05 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.602 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:19.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:19.605 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:19.617+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2523 write 0xb2a84f1 thru 0xb2aa372 (0x1e82 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2524 punch from 0x523d594 to 0x52431b6, (0x5c22 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2525 read 0xa37b17c thru 0xa38aea0 (0xfd25 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2526 punch from 0xaa714b8 to 0xaa7d816, (0xc35e bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2527 write 0xa3e212f thru 0xa3e81ee (0x60c0 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2528 read 0x2c54ac4 thru 0x2c5758b (0x2ac8 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2529 write 0x419b9f3 thru 0x41a8d98 (0xd3a6 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2530 writesame 0xb49f3b1 thru 0xb4a9f80 data_size 0x8(0xabd0 bytes) 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2531 trunc from 0xb4a9f81 to 0xe722e52 2024-04-23T23:50:19.607 INFO:teuthology.orchestra.run.smithi092.stdout:2532 write 0x14126b2 thru 0x141ad93 (0x86e2 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2533 trunc from 0xe722e52 to 0x8910107 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2534 punch from 0x73508d6 to 0x7355cfc, (0x5426 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2536 read 0x33ab9f0 thru 0x33afbf1 (0x4202 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2537 punch from 0x30fb50b to 0x3102bac, (0x76a1 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2538 write 0x7c4bbed thru 0x7c56199 (0xa5ad bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2540 punch from 0x77ac289 to 0x77b3eee, (0x7c65 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2541 trunc from 0x8910107 to 0xcdb1c50 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2542 writesame 0x3307e6 thru 0x33975c data_size 0x1(0x8f77 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2543 write 0xb59f7dc thru 0xb5acb17 (0xd33c bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2544 read 0xae00c6d thru 0xae012b1 (0x645 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2545 write 0x86e71e2 thru 0x86f1738 (0xa557 bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2548 read 0x8ec4629 thru 0x8ecd607 (0x8fdf bytes) 2024-04-23T23:50:19.608 INFO:teuthology.orchestra.run.smithi092.stdout:2549 read 0xbfbcbf3 thru 0xbfc10c0 (0x44ce bytes) 2024-04-23T23:50:19.609 INFO:teuthology.orchestra.run.smithi092.stdout:2550 trunc from 0xcdb1c50 to 0x72940b1 2024-04-23T23:50:19.615 INFO:teuthology.orchestra.run.smithi092.stdout:2551 trunc from 0x72940b1 to 0x591de61 2024-04-23T23:50:19.625 INFO:teuthology.orchestra.run.smithi092.stdout:2552 write 0x491807e thru 0x4918fdb (0xf5e bytes) 2024-04-23T23:50:19.626 INFO:teuthology.orchestra.run.smithi092.stdout:2553 read 0x1cac144 thru 0x1cb15a0 (0x545d bytes) 2024-04-23T23:50:19.626 INFO:teuthology.orchestra.run.smithi092.stdout:2554 read 0x46aa4ac thru 0x46aa883 (0x3d8 bytes) 2024-04-23T23:50:19.627 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.627 INFO:teuthology.orchestra.run.smithi092.stdout:2556 read 0x3282e74 thru 0x3283492 (0x61f bytes) 2024-04-23T23:50:19.627 INFO:teuthology.orchestra.run.smithi092.stdout:2557 read 0x51d4384 thru 0x51dad81 (0x69fe bytes) 2024-04-23T23:50:19.628 INFO:teuthology.orchestra.run.smithi092.stdout:2558 read 0x378ee3f thru 0x3790b76 (0x1d38 bytes) 2024-04-23T23:50:19.628 INFO:teuthology.orchestra.run.smithi092.stdout:2561 read 0xebe1db thru 0xec92dc (0xb102 bytes) 2024-04-23T23:50:19.630 INFO:teuthology.orchestra.run.smithi092.stdout:2563 trunc from 0x591de61 to 0x48255aa 2024-04-23T23:50:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:2566 punch from 0x19db79e to 0x19de444, (0x2ca6 bytes) 2024-04-23T23:50:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:2569 write 0xbbb2a7 thru 0xbc39a6 (0x8700 bytes) 2024-04-23T23:50:19.639 INFO:teuthology.orchestra.run.smithi092.stdout:2570 write 0x14683b6 thru 0x14755ed (0xd238 bytes) 2024-04-23T23:50:19.640 INFO:teuthology.orchestra.run.smithi092.stdout:2572 read 0x17bd2fa thru 0x17cc87e (0xf585 bytes) 2024-04-23T23:50:19.641 INFO:teuthology.orchestra.run.smithi092.stdout:2573 read 0x1bdb69e thru 0x1be072f (0x5092 bytes) 2024-04-23T23:50:19.641 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.652 INFO:teuthology.orchestra.run.smithi092.stdout:2576 trunc from 0x48255aa to 0xd6226e8 2024-04-23T23:50:19.658 INFO:teuthology.orchestra.run.smithi092.stdout:2577 punch from 0xc203245 to 0xc20b91f, (0x86da bytes) 2024-04-23T23:50:19.658 INFO:teuthology.orchestra.run.smithi092.stdout:2578 writesame 0x9f67b53 thru 0x9f74047 data_size 0x15(0xc4f5 bytes) 2024-04-23T23:50:19.660 INFO:teuthology.orchestra.run.smithi092.stdout:2580 punch from 0x1d8ddf0 to 0x1d99de5, (0xbff5 bytes) 2024-04-23T23:50:19.661 INFO:teuthology.orchestra.run.smithi092.stdout:2582 writesame 0xa676740 thru 0xa683392 data_size 0x1(0xcc53 bytes) 2024-04-23T23:50:19.694 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.694 INFO:teuthology.orchestra.run.smithi092.stdout:2585 read 0x2115093 thru 0x2120afd (0xba6b bytes) 2024-04-23T23:50:19.702 INFO:teuthology.orchestra.run.smithi092.stdout:2590 write 0xeb1e937 thru 0xeb2aaed (0xc1b7 bytes) 2024-04-23T23:50:19.704 INFO:teuthology.orchestra.run.smithi092.stdout:2591 punch from 0x795c898 to 0x7967c40, (0xb3a8 bytes) 2024-04-23T23:50:19.704 INFO:teuthology.orchestra.run.smithi092.stdout:2592 flatten 2024-04-23T23:50:19.705 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:19.718+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:19.720 INFO:teuthology.orchestra.run.smithi092.stdout:2593 trunc from 0xeb2aaee to 0x6121c30 2024-04-23T23:50:19.731 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.731 INFO:teuthology.orchestra.run.smithi092.stdout:2596 writesame 0x2cb13e9 thru 0x2cb9e3f data_size 0xf(0x8a57 bytes) 2024-04-23T23:50:19.746 INFO:teuthology.orchestra.run.smithi092.stdout:2597 write 0xbe9c406 thru 0xbea27b7 (0x63b2 bytes) 2024-04-23T23:50:19.748 INFO:teuthology.orchestra.run.smithi092.stdout:2598 write 0x818ead7 thru 0x8197394 (0x88be bytes) 2024-04-23T23:50:19.749 INFO:teuthology.orchestra.run.smithi092.stdout:2599 punch from 0x3360957 to 0x3362a4e, (0x20f7 bytes) 2024-04-23T23:50:19.749 INFO:teuthology.orchestra.run.smithi092.stdout:2600 write 0x6a0f4b8 thru 0x6a1d298 (0xdde1 bytes) 2024-04-23T23:50:19.751 INFO:teuthology.orchestra.run.smithi092.stdout:2601 punch from 0x568081e to 0x568ce23, (0xc605 bytes) 2024-04-23T23:50:19.751 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.751 INFO:teuthology.orchestra.run.smithi092.stdout:2607 write 0x1a6b169 thru 0x1a6f07f (0x3f17 bytes) 2024-04-23T23:50:19.752 INFO:teuthology.orchestra.run.smithi092.stdout:2608 write 0x69c6c68 thru 0x69d3e63 (0xd1fc bytes) 2024-04-23T23:50:19.753 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:30310 3.f 3:f24e978e:::rbd_data.11797a453d95.0000000000000005:head [write 9989224~53756 in=53756b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:19.767591+0000", 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00066269199999999999, 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068969300000000001, 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "waiting for sub ops", 2024-04-23T23:50:19.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 30310 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767591+0000", 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767591+0000", 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767592+0000", 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-07 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767653+0000", 2024-04-23T23:50:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0652999999999997e-05 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767663+0000", 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9580000000000002e-06 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767667+0000", 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8069999999999999e-06 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767717+0000", 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9240000000000003e-05 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767797+0000", 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0687999999999997e-05 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:19.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:19.767925+0000", 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012810400000000001 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 1 2024-04-23T23:50:19.759 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:19.760 INFO:teuthology.orchestra.run.smithi092.stdout:2610 read 0x2809fea thru 0x280ddc1 (0x3dd8 bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2612 write 0x2d2c5fa thru 0x2d2fdb8 (0x37bf bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2615 writesame 0x5da14ac thru 0x5dab487 data_size 0x1a(0x9fdc bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2617 read 0x47104ab thru 0x4716674 (0x61ca bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2618 punch from 0x94c7ada to 0x94d1fcd, (0xa4f3 bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2619 punch from 0xbcf66ec to 0xbd00a18, (0xa32c bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2621 read 0x1889607 thru 0x1891c65 (0x865f bytes) 2024-04-23T23:50:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:2622 write 0x70b811a thru 0x70c30c4 (0xafab bytes) 2024-04-23T23:50:19.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:19.764 INFO:teuthology.orchestra.run.smithi092.stdout:2623 trunc from 0xbea27b8 to 0x90bfb70 2024-04-23T23:50:19.773 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.773 INFO:teuthology.orchestra.run.smithi092.stdout:2625 writesame 0x2bb63cd thru 0x2bbaf13 data_size 0x1(0x4b47 bytes) 2024-04-23T23:50:19.796 INFO:teuthology.orchestra.run.smithi092.stdout:2629 writesame 0xd760fbe thru 0xd76691b data_size 0x1f(0x595e bytes) 2024-04-23T23:50:19.797 INFO:teuthology.orchestra.run.smithi092.stdout:2630 read 0x6decfcb thru 0x6dee3ab (0x13e1 bytes) 2024-04-23T23:50:19.797 INFO:teuthology.orchestra.run.smithi092.stdout:2631 read 0x8892a75 thru 0x8893515 (0xaa1 bytes) 2024-04-23T23:50:19.798 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.798 INFO:teuthology.orchestra.run.smithi092.stdout:2633 write 0x9e2e117 thru 0x9e32c15 (0x4aff bytes) 2024-04-23T23:50:19.800 INFO:teuthology.orchestra.run.smithi092.stdout:2634 trunc from 0xd76691c to 0xd8d61e9 2024-04-23T23:50:19.805 INFO:teuthology.orchestra.run.smithi092.stdout:2635 read 0xc33b0d4 thru 0xc33c18d (0x10ba bytes) 2024-04-23T23:50:19.806 INFO:teuthology.orchestra.run.smithi092.stdout:2636 read 0x30647 thru 0x34ecf (0x4889 bytes) 2024-04-23T23:50:19.808 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:19.822+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:19.814 INFO:teuthology.orchestra.run.smithi092.stdout:2637 writesame 0xeab792d thru 0xeac61bf data_size 0x1(0xe893 bytes) 2024-04-23T23:50:19.816 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.816 INFO:teuthology.orchestra.run.smithi092.stdout:2639 punch from 0xae53c9c to 0xae5f891, (0xbbf5 bytes) 2024-04-23T23:50:19.830 INFO:teuthology.orchestra.run.smithi092.stdout:2640 trunc from 0xeac61c0 to 0x23fcbdd 2024-04-23T23:50:19.842 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.842 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.842 INFO:teuthology.orchestra.run.smithi092.stdout:2646 read 0x184650d thru 0x185330a (0xcdfe bytes) 2024-04-23T23:50:19.849 INFO:teuthology.orchestra.run.smithi092.stdout:2647 write 0x332571e thru 0x332e05d (0x8940 bytes) 2024-04-23T23:50:19.851 INFO:teuthology.orchestra.run.smithi092.stdout:2648 read 0xaefaa3 thru 0xaf5143 (0x56a1 bytes) 2024-04-23T23:50:19.862 INFO:teuthology.orchestra.run.smithi092.stdout:2650 writesame 0x81f4f5c thru 0x81fd542 data_size 0x7(0x85e7 bytes) 2024-04-23T23:50:19.863 INFO:teuthology.orchestra.run.smithi092.stdout:2652 write 0x5756178 thru 0x57656e4 (0xf56d bytes) 2024-04-23T23:50:19.865 INFO:teuthology.orchestra.run.smithi092.stdout:2653 read 0x6b69dc1 thru 0x6b6c087 (0x22c7 bytes) 2024-04-23T23:50:19.875 INFO:teuthology.orchestra.run.smithi092.stdout:2654 writesame 0xba2a4dd thru 0xba2d7c3 data_size 0x1(0x32e7 bytes) 2024-04-23T23:50:19.877 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:19.878 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:19.881 INFO:teuthology.orchestra.run.smithi092.stdout:2655 write 0xc800d47 thru 0xc8027b6 (0x1a70 bytes) 2024-04-23T23:50:19.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:19.891 INFO:teuthology.orchestra.run.smithi092.stdout:2657 write 0xe68a511 thru 0xe69868e (0xe17e bytes) 2024-04-23T23:50:19.892 INFO:teuthology.orchestra.run.smithi092.stdout:2658 write 0x825028e thru 0x82524f5 (0x2268 bytes) 2024-04-23T23:50:19.901 INFO:teuthology.orchestra.run.smithi092.stdout:2659 trunc from 0xe69868f to 0x85d218d 2024-04-23T23:50:19.906 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:19.920+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:19.909 INFO:teuthology.orchestra.run.smithi092.stdout:2661 punch from 0x132867c to 0x133422e, (0xbbb2 bytes) 2024-04-23T23:50:19.909 INFO:teuthology.orchestra.run.smithi092.stdout:2662 punch from 0x34e7a68 to 0x34e9044, (0x15dc bytes) 2024-04-23T23:50:19.909 INFO:teuthology.orchestra.run.smithi092.stdout:2663 punch from 0x31ff118 to 0x3206d31, (0x7c19 bytes) 2024-04-23T23:50:19.909 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:19.909 INFO:teuthology.orchestra.run.smithi092.stdout:2666 write 0x7168ea4 thru 0x7169fec (0x1149 bytes) 2024-04-23T23:50:19.910 INFO:teuthology.orchestra.run.smithi092.stdout:2667 writesame 0x16aec25 thru 0x16bd1b2 data_size 0x1(0xe58e bytes) 2024-04-23T23:50:19.956 INFO:teuthology.orchestra.run.smithi092.stdout:2668 write 0xba7f225 thru 0xba84da3 (0x5b7f bytes) 2024-04-23T23:50:19.958 INFO:teuthology.orchestra.run.smithi092.stdout:2669 read 0x594aa41 thru 0x595072d (0x5ced bytes) 2024-04-23T23:50:19.970 INFO:teuthology.orchestra.run.smithi092.stdout:2670 trunc from 0xba84da4 to 0x14d0cdd 2024-04-23T23:50:19.982 INFO:teuthology.orchestra.run.smithi092.stdout:2671 trunc from 0x14d0cdd to 0x22d0f97 2024-04-23T23:50:19.988 INFO:teuthology.orchestra.run.smithi092.stdout:2672 punch from 0x1cbbb84 to 0x1ccbb16, (0xff92 bytes) 2024-04-23T23:50:20.007 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:20.021+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:20.007 INFO:teuthology.orchestra.run.smithi092.stdout:2673 write 0xd9d1534 thru 0xd9d66e0 (0x51ad bytes) 2024-04-23T23:50:20.018 INFO:teuthology.orchestra.run.smithi092.stdout:2675 trunc from 0xd9d66e1 to 0x57e94e5 2024-04-23T23:50:20.027 INFO:teuthology.orchestra.run.smithi092.stdout:2677 read 0xac6a8a thru 0xad281a (0xbd91 bytes) 2024-04-23T23:50:20.027 INFO:teuthology.orchestra.run.smithi092.stdout:2679 read 0x1f0f4e6 thru 0x1f15e5f (0x697a bytes) 2024-04-23T23:50:20.036 INFO:teuthology.orchestra.run.smithi092.stdout:2680 trunc from 0x57e94e5 to 0xc4c6796 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.848814615000002, 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:20.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:20.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:20.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.500077946999999, 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:20.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:20.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:20.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.500061494000001, 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.500034753, 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.500015958000001, 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.062 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.499998374, 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:20.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:20.067 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.48830287, 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:20.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:20.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.488127183, 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:20.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:20.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:20.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.487975664, 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:20.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:20.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.487693038, 2024-04-23T23:50:20.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:20.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:20.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.487547172999999, 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.087 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:20.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:20.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.486597488999999, 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.486574595, 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.095 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:20.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:20.099 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.47055922, 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:20.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:20.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.470549586000001, 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.455517224999999, 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.108 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.455495901000001, 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.455473076000001, 2024-04-23T23:50:20.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:20.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:20.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.455450397, 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.121 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:20.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.453148673999999, 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:20.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:20.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:20.130 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:20.132 INFO:teuthology.orchestra.run.smithi092.stdout:2681 punch from 0xaa6fb3e to 0xaa705ab, (0xa6d bytes) 2024-04-23T23:50:20.132 INFO:teuthology.orchestra.run.smithi092.stdout:2682 writesame 0xae1ff30 thru 0xae2aa35 data_size 0x3(0xab06 bytes) 2024-04-23T23:50:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:50:20.133 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:20.146+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:20.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:20.134 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:20.134 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:20.134 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:20.134 INFO:teuthology.orchestra.run.smithi092.stdout:2687 writesame 0xe32ac58 thru 0xe333a3d data_size 0x29(0x8de6 bytes) 2024-04-23T23:50:20.134 INFO:teuthology.orchestra.run.smithi092.stdout:2689 writesame 0x4e107b4 thru 0x4e19a5e data_size 0x1(0x92ab bytes) 2024-04-23T23:50:20.135 INFO:teuthology.orchestra.run.smithi092.stdout:2690 write 0x7c95b7f thru 0x7ca3ee0 (0xe362 bytes) 2024-04-23T23:50:20.135 INFO:teuthology.orchestra.run.smithi092.stdout:2691 trunc from 0xe333a3e to 0xa509bd0 2024-04-23T23:50:20.135 INFO:teuthology.orchestra.run.smithi092.stdout:2693 write 0x3858c0a thru 0x3862eb2 (0xa2a9 bytes) 2024-04-23T23:50:20.136 INFO:teuthology.orchestra.run.smithi092.stdout:2695 writesame 0xd826601 thru 0xd83365d data_size 0x1(0xd05d bytes) 2024-04-23T23:50:20.136 INFO:teuthology.orchestra.run.smithi092.stdout:2696 writesame 0x232ff87 thru 0x2338513 data_size 0x1(0x858d bytes) 2024-04-23T23:50:20.140 INFO:teuthology.orchestra.run.smithi092.stdout:2697 write 0xd4e9dbf thru 0xd4f85a3 (0xe7e5 bytes) 2024-04-23T23:50:20.142 INFO:teuthology.orchestra.run.smithi092.stdout:2698 write 0x4499448 thru 0x449f752 (0x630b bytes) 2024-04-23T23:50:20.143 INFO:teuthology.orchestra.run.smithi092.stdout:2701 read 0x1d881cc thru 0x1d8c58b (0x43c0 bytes) 2024-04-23T23:50:20.144 INFO:teuthology.orchestra.run.smithi092.stdout:2702 punch from 0x9d6ae7a to 0x9d6f75b, (0x48e1 bytes) 2024-04-23T23:50:20.144 INFO:teuthology.orchestra.run.smithi092.stdout:2703 punch from 0xfa6492 to 0xfaecd3, (0x8841 bytes) 2024-04-23T23:50:20.144 INFO:teuthology.orchestra.run.smithi092.stdout:2704 read 0x1d9a2bd thru 0x1da8b8f (0xe8d3 bytes) 2024-04-23T23:50:20.145 INFO:teuthology.orchestra.run.smithi092.stdout:2705 write 0x537620f thru 0x5385f16 (0xfd08 bytes) 2024-04-23T23:50:20.162 INFO:teuthology.orchestra.run.smithi092.stdout:2706 trunc from 0xd83365e to 0xd40441 2024-04-23T23:50:20.175 INFO:teuthology.orchestra.run.smithi092.stdout:2707 writesame 0xa8c41 thru 0xb6ac7 data_size 0x11(0xde87 bytes) 2024-04-23T23:50:20.193 INFO:teuthology.orchestra.run.smithi092.stdout:2708 write 0x7e89431 thru 0x7e95c17 (0xc7e7 bytes) 2024-04-23T23:50:20.206 INFO:teuthology.orchestra.run.smithi092.stdout:2709 write 0xdca66a3 thru 0xdcae0f3 (0x7a51 bytes) 2024-04-23T23:50:20.207 INFO:teuthology.orchestra.run.smithi092.stdout:2710 writesame 0x76228fb thru 0x762abe5 data_size 0x1(0x82eb bytes) 2024-04-23T23:50:20.232 INFO:teuthology.orchestra.run.smithi092.stdout:2711 punch from 0xa9cb7b0 to 0xa9d2827, (0x7077 bytes) 2024-04-23T23:50:20.233 INFO:teuthology.orchestra.run.smithi092.stdout:2712 read 0xa7a1200 thru 0xa7a55b4 (0x43b5 bytes) 2024-04-23T23:50:20.233 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:20.246+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:20.233 INFO:teuthology.orchestra.run.smithi092.stdout:2713 punch from 0xba8b551 to 0xba9092d, (0x53dc bytes) 2024-04-23T23:50:20.234 INFO:teuthology.orchestra.run.smithi092.stdout:2714 punch from 0xd51cdf3 to 0xd529868, (0xca75 bytes) 2024-04-23T23:50:20.234 INFO:teuthology.orchestra.run.smithi092.stdout:2715 read 0xcdd38dd thru 0xcde1e89 (0xe5ad bytes) 2024-04-23T23:50:20.234 INFO:teuthology.orchestra.run.smithi092.stdout:2716 writesame 0xa50d147 thru 0xa515571 data_size 0x5(0x842b bytes) 2024-04-23T23:50:20.240 INFO:teuthology.orchestra.run.smithi092.stdout:2717 read 0x6e7fd53 thru 0x6e835d0 (0x387e bytes) 2024-04-23T23:50:20.240 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:20.240 INFO:teuthology.orchestra.run.smithi092.stdout:2719 punch from 0x54084a2 to 0x54129cf, (0xa52d bytes) 2024-04-23T23:50:20.241 INFO:teuthology.orchestra.run.smithi092.stdout:2720 read 0xbbbdb01 thru 0xbbc9489 (0xb989 bytes) 2024-04-23T23:50:20.241 INFO:teuthology.orchestra.run.smithi092.stdout:2721 write 0x1476203 thru 0x147af8b (0x4d89 bytes) 2024-04-23T23:50:20.243 INFO:teuthology.orchestra.run.smithi092.stdout:2722 read 0x76e1171 thru 0x76e1a40 (0x8d0 bytes) 2024-04-23T23:50:20.243 INFO:teuthology.orchestra.run.smithi092.stdout:2723 read 0x6d1d0c5 thru 0x6d20d51 (0x3c8d bytes) 2024-04-23T23:50:20.250 INFO:teuthology.orchestra.run.smithi092.stdout:2724 trunc from 0xdcae0f4 to 0x83967e7 2024-04-23T23:50:20.260 INFO:teuthology.orchestra.run.smithi092.stdout:2727 read 0x32e2372 thru 0x32ed9fc (0xb68b bytes) 2024-04-23T23:50:20.260 INFO:teuthology.orchestra.run.smithi092.stdout:2728 read 0x2818d24 thru 0x2822a1e (0x9cfb bytes) 2024-04-23T23:50:20.261 INFO:teuthology.orchestra.run.smithi092.stdout:2730 read 0x3d34b71 thru 0x3d447c9 (0xfc59 bytes) 2024-04-23T23:50:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:20.266 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:20.266 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:2731 trunc from 0x83967e7 to 0x3d44d9b 2024-04-23T23:50:20.274 INFO:teuthology.orchestra.run.smithi092.stdout:2732 clone 17 order 21 su 262144 sc 11 2024-04-23T23:50:20.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:20.360 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:20.374+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:20.418 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:20.419 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:20.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:20.435 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:20.449+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:20.536 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:20.550+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:20.571 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10283 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7225344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311444400999999, 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035444584000000001, 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10283 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273272+0000", 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273274+0000", 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273279+0000", 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5820000000000001e-06 2024-04-23T23:50:20.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273284+0000", 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308551+0000", 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035267169000000001 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308626+0000", 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4328999999999999e-05 2024-04-23T23:50:20.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308716+0000", 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0647000000000001e-05 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10284 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7389184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311426365999999, 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035618043000000002, 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10284 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273290+0000", 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273292+0000", 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9360000000000002e-06 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273298+0000", 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6860000000000003e-06 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273303+0000", 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1830000000000002e-06 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308748+0000", 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035444749999999997 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308824+0000", 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6705999999999994e-05 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308908+0000", 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10285 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:20.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311397747999999, 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035744949999999998, 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10285 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273318+0000", 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273319+0000", 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6000000000000004e-07 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273322+0000", 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.236e-06 2024-04-23T23:50:20.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273329+0000", 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1350000000000003e-06 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273335+0000", 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5149999999999997e-06 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.308939+0000", 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035603856000000003 2024-04-23T23:50:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309006+0000", 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6934999999999995e-05 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309063+0000", 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7713000000000001e-05 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10286 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311371728999999, 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035978035999999998, 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10286 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273344+0000", 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273345+0000", 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273347+0000", 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.125e-06 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273363+0000", 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6353000000000001e-05 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273370+0000", 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309119+0000", 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035748874 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309212+0000", 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2523999999999997e-05 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309322+0000", 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110597 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10287 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8142848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311337425, 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036157685000000002, 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10287 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273379+0000", 2024-04-23T23:50:20.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273382+0000", 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4959999999999999e-06 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273389+0000", 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9160000000000004e-06 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273394+0000", 2024-04-23T23:50:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1710000000000001e-06 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309383+0000", 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035989128000000002 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309448+0000", 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5419999999999994e-05 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309536+0000", 2024-04-23T23:50:20.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8034000000000005e-05 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10288 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8306688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311305129000001, 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036258020000000002, 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10288 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273411+0000", 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273414+0000", 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7259999999999998e-06 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273422+0000", 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5390000000000002e-06 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273427+0000", 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309563+0000", 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036135689999999998 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309618+0000", 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4994000000000002e-05 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309669+0000", 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1147999999999999e-05 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10289 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8470528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311271960999999, 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036365322999999998, 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10289 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273444+0000", 2024-04-23T23:50:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273445+0000", 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273447+0000", 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.587e-06 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273456+0000", 2024-04-23T23:50:20.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6770000000000001e-06 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273462+0000", 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8810000000000001e-06 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309690+0000", 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036228306000000002 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309753+0000", 2024-04-23T23:50:20.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2772000000000001e-05 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309809+0000", 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6560000000000001e-05 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10290 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311246643, 2024-04-23T23:50:20.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036521275999999998, 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10290 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273469+0000", 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273470+0000", 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-07 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273472+0000", 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2330000000000001e-06 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273479+0000", 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4080000000000003e-06 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273484+0000", 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9819999999999999e-06 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309838+0000", 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036354356999999997 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309923+0000", 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5130000000000007e-05 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.309991+0000", 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7630000000000001e-05 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10291 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9060352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311223563, 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036628916999999997, 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10291 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273493+0000", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0900000000000002e-07 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273495+0000", 2024-04-23T23:50:20.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3190000000000002e-06 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273502+0000", 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4609999999999996e-06 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273507+0000", 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9490000000000004e-06 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310011+0000", 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036504395000000002 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310057+0000", 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6008999999999999e-05 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310121+0000", 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4275000000000004e-05 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10292 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9224192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311171922, 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036703116000000001, 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10292 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273544+0000", 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273545+0000", 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273548+0000", 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273557+0000", 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2979999999999997e-06 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273563+0000", 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7810000000000002e-06 2024-04-23T23:50:20.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310148+0000", 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036585212999999998 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310198+0000", 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0686999999999998e-05 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310247+0000", 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8801000000000003e-05 2024-04-23T23:50:20.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10293 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9388032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311065130999999, 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036741629999999997, 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10293 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273651+0000", 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273652+0000", 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273665+0000", 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.34e-05 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273673+0000", 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3070000000000003e-06 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273679+0000", 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8610000000000003e-06 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310264+0000", 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036584942000000002 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310304+0000", 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9932999999999999e-05 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310393+0000", 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8640999999999994e-05 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10294 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9551872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311028463, 2024-04-23T23:50:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036864814000000003, 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10294 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273688+0000", 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273690+0000", 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8139999999999999e-06 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273696+0000", 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7760000000000003e-06 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273701+0000", 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0159999999999999e-06 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310423+0000", 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036722312999999999 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310489+0000", 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5741000000000006e-05 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310552+0000", 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3674000000000001e-05 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10295 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 9977856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.311008212999999, 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036991522999999998, 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10295 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273708+0000", 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.77e-07 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273710+0000", 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0829999999999998e-06 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273716+0000", 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9410000000000004e-06 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273721+0000", 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.955e-06 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310584+0000", 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036862930000000002 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310644+0000", 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0037000000000001e-05 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310699+0000", 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.52e-05 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10296 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10141696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310986352, 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037192338999999998, 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10296 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273730+0000", 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-04-23T23:50:20.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273732+0000", 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273739+0000", 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5599999999999999e-06 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273744+0000", 2024-04-23T23:50:20.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0980000000000001e-06 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310736+0000", 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036992522999999999 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310811+0000", 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4184999999999995e-05 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310922+0000", 2024-04-23T23:50:20.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111532 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10297 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10305536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310955224000001, 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037380582000000002, 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10297 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273761+0000", 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273764+0000", 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273790+0000", 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273797+0000", 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0129999999999996e-06 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.310950+0000", 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037152875000000002 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311040+0000", 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9816999999999997e-05 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311141+0000", 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010139700000000001 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10298 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10469376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310909983, 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037503460000000002, 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10298 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273806+0000", 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273807+0000", 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8800000000000003e-07 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273809+0000", 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273816+0000", 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5830000000000003e-06 2024-04-23T23:50:20.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273821+0000", 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.848e-06 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311188+0000", 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037366982 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311234+0000", 2024-04-23T23:50:20.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6269e-05 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311310+0000", 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780999999999998e-05 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10299 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 10895360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310888883000001, 2024-04-23T23:50:20.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037660666000000002, 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10299 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273827+0000", 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273828+0000", 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273835+0000", 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-23T23:50:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273844+0000", 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1360000000000003e-06 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273850+0000", 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3360000000000002e-06 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311345+0000", 2024-04-23T23:50:20.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037495406000000002 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311415+0000", 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9605999999999997e-05 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311488+0000", 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3161e-05 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10300 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11059200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310825314000001, 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037811175000000002, 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10300 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273891+0000", 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2200000000000004e-07 2024-04-23T23:50:20.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273894+0000", 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9550000000000001e-06 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273903+0000", 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3299999999999999e-06 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273910+0000", 2024-04-23T23:50:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8689999999999998e-06 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311574+0000", 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037664133000000002 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311616+0000", 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2481000000000002e-05 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311702+0000", 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5785000000000006e-05 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10301 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11223040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310798189, 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037936401000000002, 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10301 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:20.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273918+0000", 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.44e-07 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273921+0000", 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273927+0000", 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5270000000000004e-06 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273932+0000", 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.215e-06 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311737+0000", 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037804799 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311801+0000", 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3652999999999995e-05 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311854+0000", 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3542999999999999e-05 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:10302 3.1b 3:dc43a518:::rbd_data.117916e6274d.0000000000000004:c [read 11386880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:20.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.310775034000001, 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038084931000000002, 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 10302 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273941+0000", 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273944+0000", 2024-04-23T23:50:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273950+0000", 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6540000000000002e-06 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.273956+0000", 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311874+0000", 2024-04-23T23:50:20.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037918232000000003 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.311951+0000", 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6701999999999998e-05 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:07.312026+0000", 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5469000000000005e-05 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:20.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:20.657 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:20.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:20.658 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:20.672+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:20.759 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:20.773+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:20.803 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:20.803 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:20.803 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:20.803 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:20.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:20.860 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:20.874+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:20.913 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:20.913 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:20.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:20.965 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:20.979+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:21.025 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:3037 3.4 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head [stat,truncate 5242880] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.581221500000002, 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089969830000000001, 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 3037 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456900+0000", 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0860000000000001e-06 2024-04-23T23:50:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456911+0000", 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1467e-05 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456922+0000", 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380999999999999e-05 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456929+0000", 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8000000000000001e-06 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459092+0000", 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002163697 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459193+0000", 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100709 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459396+0000", 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020303399999999999 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.462879+0000", 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034833899999999998 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465827+0000", 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029476889999999999 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465863+0000", 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5503000000000002e-05 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465896+0000", 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3226999999999999e-05 2024-04-23T23:50:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6867 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.730041091, 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077717510000000004, 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6867 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308080+0000", 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308082+0000", 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308088+0000", 2024-04-23T23:50:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308092+0000", 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6140000000000002e-06 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315699+0000", 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076064569999999996 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315761+0000", 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2194999999999997e-05 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315851+0000", 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0109000000000001e-05 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6868 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.730022010999999, 2024-04-23T23:50:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079224340000000008, 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6868 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308099+0000", 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4099999999999999e-07 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308101+0000", 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9080000000000002e-06 2024-04-23T23:50:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308106+0000", 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6869999999999999e-06 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308111+0000", 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315874+0000", 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077625799999999998 2024-04-23T23:50:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315936+0000", 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2442999999999998e-05 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316021+0000", 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4592999999999995e-05 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6871 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.730003345, 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080708470000000008, 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6871 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-23T23:50:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308119+0000", 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8509999999999999e-06 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308128+0000", 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2509999999999994e-06 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308133+0000", 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-04-23T23:50:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316044+0000", 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079107259999999999 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316108+0000", 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3937999999999997e-05 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316188+0000", 2024-04-23T23:50:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0208000000000002e-05 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6872 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729981716999999, 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082303659999999994, 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6872 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308142+0000", 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.79e-06 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308147+0000", 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3639999999999998e-06 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308151+0000", 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.34e-06 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316212+0000", 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080606099999999993 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316280+0000", 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7692000000000002e-05 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316369+0000", 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9148999999999996e-05 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6875 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729963027, 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083900060000000002, 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6875 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308158+0000", 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308160+0000", 2024-04-23T23:50:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9290000000000001e-06 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308166+0000", 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6510000000000004e-06 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308171+0000", 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2640000000000002e-06 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316398+0000", 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082269720000000008 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316466+0000", 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8361999999999994e-05 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316547+0000", 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1311999999999997e-05 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6876 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729943227, 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085708390000000002, 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6876 2024-04-23T23:50:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308178+0000", 2024-04-23T23:50:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308179+0000", 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.844e-06 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308185+0000", 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.198e-06 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308189+0000", 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1699999999999999e-06 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316574+0000", 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083852639999999999 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316652+0000", 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8078999999999996e-05 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316748+0000", 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5784000000000003e-05 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6878 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729923389, 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087314869999999996, 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6878 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308199+0000", 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0559999999999999e-06 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308205+0000", 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8329999999999999e-06 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308210+0000", 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4519999999999999e-06 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316773+0000", 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085637890000000005 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316846+0000", 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2389000000000001e-05 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316928+0000", 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2602000000000006e-05 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6880 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729904998, 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089037119999999994, 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6880 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308216+0000", 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308224+0000", 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4409999999999993e-06 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308230+0000", 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8340000000000004e-06 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308234+0000", 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4109999999999998e-06 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316953+0000", 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087181989999999994 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317029+0000", 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6186999999999994e-05 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317119+0000", 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0286999999999998e-05 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6883 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729869249, 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090432810000000002, 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6883 2024-04-23T23:50:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308252+0000", 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4199999999999995e-07 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308254+0000", 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4940000000000002e-06 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308262+0000", 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3780000000000003e-06 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308267+0000", 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8540000000000002e-06 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317145+0000", 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088778469999999995 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317212+0000", 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6686000000000005e-05 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317294+0000", 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2379999999999994e-05 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6884 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729846569999999, 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092106370000000007, 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6884 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2500000000000001e-07 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308276+0000", 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.978e-06 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308282+0000", 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308286+0000", 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3649999999999997e-06 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317320+0000", 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009034172 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317394+0000", 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3825000000000006e-05 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317484+0000", 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0340000000000006e-05 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6886 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729828749999999, 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093837590000000002, 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6886 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308294+0000", 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.86e-06 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308300+0000", 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308305+0000", 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3249999999999998e-06 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317511+0000", 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092054749999999994 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317587+0000", 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6179000000000004e-05 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317675+0000", 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8595000000000001e-05 2024-04-23T23:50:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6890 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729809648, 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095370109999999998, 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6890 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308313+0000", 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8989999999999999e-06 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308320+0000", 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.827e-06 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308324+0000", 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2830000000000003e-06 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317700+0000", 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093762359999999996 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317767+0000", 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6550000000000005e-05 2024-04-23T23:50:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317848+0000", 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0844e-05 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6892 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729790775, 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097323389999999996, 2024-04-23T23:50:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6892 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308332+0000", 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.883e-06 2024-04-23T23:50:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308338+0000", 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9680000000000003e-06 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308342+0000", 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.143e-06 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317873+0000", 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095314449999999995 2024-04-23T23:50:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317937+0000", 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3165999999999998e-05 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318062+0000", 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012526900000000001 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6893 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729773257, 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099354839999999996, 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6893 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308348+0000", 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0399999999999996e-07 2024-04-23T23:50:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308349+0000", 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7540000000000001e-06 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308355+0000", 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3419999999999998e-06 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308359+0000", 2024-04-23T23:50:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4959999999999998e-06 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318107+0000", 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097478240000000004 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318189+0000", 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1762999999999996e-05 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318283+0000", 2024-04-23T23:50:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3801000000000005e-05 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6894 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729755859000001, 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126606, 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6894 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308365+0000", 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308367+0000", 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8920000000000001e-06 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308373+0000", 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8939999999999998e-06 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308377+0000", 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2529999999999998e-06 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318324+0000", 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099474309999999996 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318393+0000", 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8727999999999997e-05 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318491+0000", 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8036999999999997e-05 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6897 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729738239, 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010295080999999999, 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6897 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308383+0000", 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308384+0000", 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.832e-06 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308396+0000", 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1208e-05 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308400+0000", 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.639e-06 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318519+0000", 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010119025 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318592+0000", 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3124e-05 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318677+0000", 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4792999999999999e-05 2024-04-23T23:50:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6898 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729714101999999, 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010449478999999999, 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6898 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308407+0000", 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308409+0000", 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308414+0000", 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5230000000000003e-06 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308418+0000", 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2039999999999999e-06 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318706+0000", 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01028717 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318774+0000", 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7983000000000003e-05 2024-04-23T23:50:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318856+0000", 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2047999999999998e-05 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6900 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729696203, 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010615769000000001, 2024-04-23T23:50:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6900 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308427+0000", 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1789999999999998e-06 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308432+0000", 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5990000000000001e-06 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308440+0000", 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318884+0000", 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010443913000000001 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318953+0000", 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8650000000000002e-05 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319040+0000", 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6908000000000002e-05 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6902 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.729673215, 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010766184, 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6902 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308455+0000", 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0239999999999997e-06 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308462+0000", 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-06 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308471+0000", 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4509999999999998e-06 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319063+0000", 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010591685999999999 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319132+0000", 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8628999999999996e-05 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319213+0000", 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1453999999999996e-05 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:21.118 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:21.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:21.120 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:21.134+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:21.166 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-04-23T23:50:21.221 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:21.234+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:21.312 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:21.312 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:21.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:21.322 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:21.335+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:21.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:21.404 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-04-23T23:50:21.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:21.404 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-04-23T23:50:21.404 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-23T23:50:21.404 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-23T23:50:21.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-04-23T23:50:21.448 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:21.462+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:21.463 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:21.469 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:21.470 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:21.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:21.482 INFO:teuthology.orchestra.run.smithi092.stdout:2736 write 0xc86962d thru 0xc874743 (0xb117 bytes) 2024-04-23T23:50:21.485 INFO:teuthology.orchestra.run.smithi092.stdout:2737 write 0x84d1ec3 thru 0x84dde74 (0xbfb2 bytes) 2024-04-23T23:50:21.488 INFO:teuthology.orchestra.run.smithi092.stdout:2739 write 0xc04448f thru 0xc054347 (0xfeb9 bytes) 2024-04-23T23:50:21.491 INFO:teuthology.orchestra.run.smithi092.stdout:2742 write 0x612431e thru 0x61297c8 (0x54ab bytes) 2024-04-23T23:50:21.496 INFO:teuthology.orchestra.run.smithi092.stdout:2743 trunc from 0xc874744 to 0xac9cbcc 2024-04-23T23:50:21.506 INFO:teuthology.orchestra.run.smithi092.stdout:2744 writesame 0x8cba4af thru 0x8cc718d data_size 0x1(0xccdf bytes) 2024-04-23T23:50:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-23T23:50:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:21.523 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:21.536+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:21.534 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:21.541 INFO:teuthology.orchestra.run.smithi092.stdout:2745 punch from 0x961a274 to 0x9620138, (0x5ec4 bytes) 2024-04-23T23:50:21.553 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:21.553 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:21.554 INFO:teuthology.orchestra.run.smithi092.stdout:2746 trunc from 0xac9cbcc to 0x389b76 2024-04-23T23:50:21.576 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:21.576 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:21.576 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:21.588 INFO:teuthology.orchestra.run.smithi092.stdout:2748 write 0x1f377f4 thru 0x1f46e7e (0xf68b bytes) 2024-04-23T23:50:21.594 INFO:teuthology.orchestra.run.smithi092.stdout:2749 clone 18 order 22 su 1048576 sc 9 2024-04-23T23:50:21.596 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:21.597 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:21.605 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:21.605 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:21.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:21.615 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:21.615 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:21.648 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:21.649 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:21.652 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:21.652 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:21.659 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:21.673+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:21.719 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:21.719 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:21.726 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:21.740+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:21.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:21.827 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:21.841+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:21.871 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:21.871 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:21.879 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:21.931 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:21.945+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:22.021 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.022 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:22.031 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:22.045+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:22.122 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone17 from 0x1f46e7f (overlap 0x389b76) to 0x27a005 2024-04-23T23:50:22.132 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:22.145+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:22.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:50:22.145 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.145 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:22.181 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-04-23T23:50:22.233 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:22.246+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:22.258 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.258 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.265 INFO:teuthology.orchestra.run.smithi092.stdout:2750 read 0x77b811 thru 0x780f05 (0x56f5 bytes) 2024-04-23T23:50:22.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:22.281 INFO:teuthology.orchestra.run.smithi092.stdout:2751 trunc from 0x1f46e7f to 0xe8467b9 2024-04-23T23:50:22.305 INFO:teuthology.orchestra.run.smithi092.stdout:2752 trunc from 0xe8467b9 to 0x1b6b35f 2024-04-23T23:50:22.333 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:22.347+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:22.344 INFO:teuthology.orchestra.run.smithi092.stdout:2753 write 0xc2158fd thru 0xc223294 (0xd998 bytes) 2024-04-23T23:50:22.347 INFO:teuthology.orchestra.run.smithi092.stdout:2754 writesame 0x4d22a31 thru 0x4d311f1 data_size 0x1(0xe7c1 bytes) 2024-04-23T23:50:22.378 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.378 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.379 INFO:teuthology.orchestra.run.smithi092.stdout:2755 read 0x506a621 thru 0x5070b63 (0x6543 bytes) 2024-04-23T23:50:22.380 INFO:teuthology.orchestra.run.smithi092.stdout:2756 writesame 0x5fe22bf thru 0x5fe93dd data_size 0x31(0x711f bytes) 2024-04-23T23:50:22.382 INFO:teuthology.orchestra.run.smithi092.stdout:2757 punch from 0x44be17c to 0x44c8187, (0xa00b bytes) 2024-04-23T23:50:22.383 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:22.383 INFO:teuthology.orchestra.run.smithi092.stdout:2759 write 0x448ea2a thru 0x4493abc (0x5093 bytes) 2024-04-23T23:50:22.385 INFO:teuthology.orchestra.run.smithi092.stdout:2760 read 0x270be5 thru 0x27c651 (0xba6d bytes) 2024-04-23T23:50:22.386 INFO:teuthology.orchestra.run.smithi092.stdout:2762 read 0x1de7659 thru 0x1df3d1a (0xc6c2 bytes) 2024-04-23T23:50:22.387 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:22.393 INFO:teuthology.orchestra.run.smithi092.stdout:2763 writesame 0xd0313b9 thru 0xd034053 data_size 0x1(0x2c9b bytes) 2024-04-23T23:50:22.396 INFO:teuthology.orchestra.run.smithi092.stdout:2764 trunc from 0xd034054 to 0xa53b70d 2024-04-23T23:50:22.407 INFO:teuthology.orchestra.run.smithi092.stdout:2766 write 0x103cfba thru 0x1040a7d (0x3ac4 bytes) 2024-04-23T23:50:22.408 INFO:teuthology.orchestra.run.smithi092.stdout:2767 read 0x9ae97cf thru 0x9af96d3 (0xff05 bytes) 2024-04-23T23:50:22.408 INFO:teuthology.orchestra.run.smithi092.stdout:2768 write 0x30aa605 thru 0x30ae592 (0x3f8e bytes) 2024-04-23T23:50:22.410 INFO:teuthology.orchestra.run.smithi092.stdout:2769 read 0xe47495 thru 0xe4d368 (0x5ed4 bytes) 2024-04-23T23:50:22.411 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:22.411 INFO:teuthology.orchestra.run.smithi092.stdout:2773 punch from 0x3101bc4 to 0x310e6ff, (0xcb3b bytes) 2024-04-23T23:50:22.411 INFO:teuthology.orchestra.run.smithi092.stdout:2774 write 0x3069b95 thru 0x306fd5f (0x61cb bytes) 2024-04-23T23:50:22.412 INFO:teuthology.orchestra.run.smithi092.stdout:2775 read 0x10e7629 thru 0x10f1bda (0xa5b2 bytes) 2024-04-23T23:50:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:2776 flatten 2024-04-23T23:50:22.419 INFO:teuthology.orchestra.run.smithi092.stdout:2777 read 0x8704b58 thru 0x8705872 (0xd1b bytes) 2024-04-23T23:50:22.419 INFO:teuthology.orchestra.run.smithi092.stdout:2778 writesame 0x8a63ae4 thru 0x8a65975 data_size 0x1(0x1e92 bytes) 2024-04-23T23:50:22.426 INFO:teuthology.orchestra.run.smithi092.stdout:2779 write 0x4330b0 thru 0x441e9e (0xedef bytes) 2024-04-23T23:50:22.427 INFO:teuthology.orchestra.run.smithi092.stdout:2781 writesame 0x18199ae thru 0x18220b9 data_size 0x2b(0x870c bytes) 2024-04-23T23:50:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:2782 write 0x443c0d1 thru 0x443c381 (0x2b1 bytes) 2024-04-23T23:50:22.434 INFO:teuthology.orchestra.run.smithi092.stdout:2783 trunc from 0xa53b70d to 0x763b48b 2024-04-23T23:50:22.440 INFO:teuthology.orchestra.run.smithi092.stdout:2784 writesame 0x5403216 thru 0x5407c19 data_size 0x6(0x4a04 bytes) 2024-04-23T23:50:22.444 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:22.445 INFO:teuthology.orchestra.run.smithi092.stdout:2787 punch from 0xf0dd77 to 0xf1536d, (0x75f6 bytes) 2024-04-23T23:50:22.445 INFO:teuthology.orchestra.run.smithi092.stdout:2788 read 0x68c2a86 thru 0x68cb162 (0x86dd bytes) 2024-04-23T23:50:22.445 INFO:teuthology.orchestra.run.smithi092.stdout:2789 read 0xc55aa2 thru 0xc5b489 (0x59e8 bytes) 2024-04-23T23:50:22.445 INFO:teuthology.orchestra.run.smithi092.stdout:2790 read 0x1a648a2 thru 0x1a6c04d (0x77ac bytes) 2024-04-23T23:50:22.453 INFO:teuthology.orchestra.run.smithi092.stdout:2791 trunc from 0x763b48b to 0xca0b620 2024-04-23T23:50:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:2795 read 0x929ddd6 thru 0x929f1b7 (0x13e2 bytes) 2024-04-23T23:50:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:2796 punch from 0x6a3bb2 to 0x6af2f0, (0xb73e bytes) 2024-04-23T23:50:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:2799 punch from 0xb5dfefc to 0xb5e08a5, (0x9a9 bytes) 2024-04-23T23:50:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:2800 write 0x8722e7a thru 0x872f749 (0xc8d0 bytes) 2024-04-23T23:50:22.461 INFO:teuthology.orchestra.run.smithi092.stdout:2801 read 0x2c765a4 thru 0x2c7eda1 (0x87fe bytes) 2024-04-23T23:50:22.466 INFO:teuthology.orchestra.run.smithi092.stdout:2802 writesame 0xce3cddc thru 0xce3d4e5 data_size 0x11(0x70a bytes) 2024-04-23T23:50:22.466 INFO:teuthology.orchestra.run.smithi092.stdout:2805 read 0x7e5197d thru 0x7e5d8a3 (0xbf27 bytes) 2024-04-23T23:50:22.466 INFO:teuthology.orchestra.run.smithi092.stdout:2807 writesame 0x2a7df41 thru 0x2a86742 data_size 0x15(0x8802 bytes) 2024-04-23T23:50:22.470 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:22.470 INFO:teuthology.orchestra.run.smithi092.stdout:2809 write 0x9e96c82 thru 0x9ea6b08 (0xfe87 bytes) 2024-04-23T23:50:22.472 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:22.486+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:22.473 INFO:teuthology.orchestra.run.smithi092.stdout:2810 write 0x6e645a5 thru 0x6e68691 (0x40ed bytes) 2024-04-23T23:50:22.476 INFO:teuthology.orchestra.run.smithi092.stdout:2812 write 0x5d0d47 thru 0x5d8aba (0x7d74 bytes) 2024-04-23T23:50:22.484 INFO:teuthology.orchestra.run.smithi092.stdout:2813 write 0xe349c93 thru 0xe35437b (0xa6e9 bytes) 2024-04-23T23:50:22.486 INFO:teuthology.orchestra.run.smithi092.stdout:2814 write 0x59d2d6 thru 0x59f27a (0x1fa5 bytes) 2024-04-23T23:50:22.488 INFO:teuthology.orchestra.run.smithi092.stdout:2815 write 0x67a457d thru 0x67a5bee (0x1672 bytes) 2024-04-23T23:50:22.492 INFO:teuthology.orchestra.run.smithi092.stdout:2817 read 0xcd8820f thru 0xcd9617c (0xdf6e bytes) 2024-04-23T23:50:22.492 INFO:teuthology.orchestra.run.smithi092.stdout:2820 punch from 0x754b98c to 0x755ae94, (0xf508 bytes) 2024-04-23T23:50:22.492 INFO:teuthology.orchestra.run.smithi092.stdout:2821 read 0x8fd9b8b thru 0x8fe481f (0xac95 bytes) 2024-04-23T23:50:22.493 INFO:teuthology.orchestra.run.smithi092.stdout:2822 read 0x4736f2a thru 0x473b415 (0x44ec bytes) 2024-04-23T23:50:22.493 INFO:teuthology.orchestra.run.smithi092.stdout:2823 write 0xaebe42c thru 0xaeccbec (0xe7c1 bytes) 2024-04-23T23:50:22.496 INFO:teuthology.orchestra.run.smithi092.stdout:2824 write 0xb1778f3 thru 0xb185408 (0xdb16 bytes) 2024-04-23T23:50:22.499 INFO:teuthology.orchestra.run.smithi092.stdout:2825 writesame 0x1a06c53 thru 0x1a0d5a0 data_size 0x3(0x694e bytes) 2024-04-23T23:50:22.505 INFO:teuthology.orchestra.run.smithi092.stdout:2830 punch from 0x1b3b088 to 0x1b3c19e, (0x1116 bytes) 2024-04-23T23:50:22.505 INFO:teuthology.orchestra.run.smithi092.stdout:2831 writesame 0x3d83efd thru 0x3d8d264 data_size 0x8(0x9368 bytes) 2024-04-23T23:50:22.510 INFO:teuthology.orchestra.run.smithi092.stdout:2832 write 0x58ed4ee thru 0x58fa64f (0xd162 bytes) 2024-04-23T23:50:22.515 INFO:teuthology.orchestra.run.smithi092.stdout:2833 read 0x4fe5463 thru 0x4fe6d40 (0x18de bytes) 2024-04-23T23:50:22.515 INFO:teuthology.orchestra.run.smithi092.stdout:2836 writesame 0xd24dc1d thru 0xd2532f6 data_size 0x1(0x56da bytes) 2024-04-23T23:50:22.532 INFO:teuthology.orchestra.run.smithi092.stdout:2837 trunc from 0xe35437c to 0xa03b76c 2024-04-23T23:50:22.535 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:22.549+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:22.537 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.538 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:22.555 INFO:teuthology.orchestra.run.smithi092.stdout:2838 write 0xa615e98 thru 0xa61d7b7 (0x7920 bytes) 2024-04-23T23:50:22.559 INFO:teuthology.orchestra.run.smithi092.stdout:2839 writesame 0x55b0689 thru 0x55bf754 data_size 0x2c(0xf0cc bytes) 2024-04-23T23:50:22.562 INFO:teuthology.orchestra.run.smithi092.stdout:2840 write 0x45dd4d5 thru 0x45ece25 (0xf951 bytes) 2024-04-23T23:50:22.563 INFO:teuthology.orchestra.run.smithi092.stdout:2841 punch from 0x873e716 to 0x87427ad, (0x4097 bytes) 2024-04-23T23:50:22.563 INFO:teuthology.orchestra.run.smithi092.stdout:2842 writesame 0x76be70b thru 0x76bf6cb data_size 0x1(0xfc1 bytes) 2024-04-23T23:50:22.579 INFO:teuthology.orchestra.run.smithi092.stdout:2844 trunc from 0xa61d7b8 to 0x1573d37 2024-04-23T23:50:22.592 INFO:teuthology.orchestra.run.smithi092.stdout:2845 clone 19 order 18 su 2048 sc 2 2024-04-23T23:50:22.636 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:22.650+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:22.737 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:22.751+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:22.799 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.799 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:22.852 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:22.866+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:22.911 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:22.911 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:22.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:22.939 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:22.953+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:23.040 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:23.054+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:23.062 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:23.063 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:23.063 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:50:23.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:23.154 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-04-23T23:50:23.159 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:23.172+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:23.179 INFO:teuthology.orchestra.run.smithi092.stdout:2846 write 0xab21bab thru 0xab24776 (0x2bcc bytes) 2024-04-23T23:50:23.183 INFO:teuthology.orchestra.run.smithi092.stdout:2847 write 0x77c1fce thru 0x77c2607 (0x63a bytes) 2024-04-23T23:50:23.187 INFO:teuthology.orchestra.run.smithi092.stdout:2850 write 0x50f2adf thru 0x50f7c33 (0x5155 bytes) 2024-04-23T23:50:23.192 INFO:teuthology.orchestra.run.smithi092.stdout:2855 write 0x4d93fb7 thru 0x4d9bfe5 (0x802f bytes) 2024-04-23T23:50:23.198 INFO:teuthology.orchestra.run.smithi092.stdout:2856 writesame 0x594bb28 thru 0x5956040 data_size 0x4f(0xa519 bytes) 2024-04-23T23:50:23.201 INFO:teuthology.orchestra.run.smithi092.stdout:2857 read 0x1f18bb9 thru 0x1f1d1cd (0x4615 bytes) 2024-04-23T23:50:23.201 INFO:teuthology.orchestra.run.smithi092.stdout:2858 read 0x41046ae thru 0x410e4ec (0x9e3f bytes) 2024-04-23T23:50:23.201 INFO:teuthology.orchestra.run.smithi092.stdout:2860 write 0x66ddd5e thru 0x66e0e5a (0x30fd bytes) 2024-04-23T23:50:23.204 INFO:teuthology.orchestra.run.smithi092.stdout:2861 read 0x5a1f598 thru 0x5a27cbe (0x8727 bytes) 2024-04-23T23:50:23.204 INFO:teuthology.orchestra.run.smithi092.stdout:2862 read 0x74c9270 thru 0x74cebdc (0x596d bytes) 2024-04-23T23:50:23.204 INFO:teuthology.orchestra.run.smithi092.stdout:2865 writesame 0x506c18b thru 0x5078e74 data_size 0x3(0xccea bytes) 2024-04-23T23:50:23.226 INFO:teuthology.orchestra.run.smithi092.stdout:2866 write 0xdf4012d thru 0xdf4e265 (0xe139 bytes) 2024-04-23T23:50:23.227 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:23.227 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:23.230 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:23.230 INFO:teuthology.orchestra.run.smithi092.stdout:2868 write 0x4e8fef8 thru 0x4e982c9 (0x83d2 bytes) 2024-04-23T23:50:23.234 INFO:teuthology.orchestra.run.smithi092.stdout:2869 punch from 0x53d1828 to 0x53ddde3, (0xc5bb bytes) 2024-04-23T23:50:23.235 INFO:teuthology.orchestra.run.smithi092.stdout:2870 read 0x8ec25ee thru 0x8ec69d8 (0x43eb bytes) 2024-04-23T23:50:23.235 INFO:teuthology.orchestra.run.smithi092.stdout:2871 punch from 0x3db7f3d to 0x3dbb077, (0x313a bytes) 2024-04-23T23:50:23.235 INFO:teuthology.orchestra.run.smithi092.stdout:2873 punch from 0x5b4804a to 0x5b4c9f2, (0x49a8 bytes) 2024-04-23T23:50:23.235 INFO:teuthology.orchestra.run.smithi092.stdout:2874 punch from 0x7283562 to 0x72860af, (0x2b4d bytes) 2024-04-23T23:50:23.235 INFO:teuthology.orchestra.run.smithi092.stdout:2875 punch from 0xcf2daf9 to 0xcf32349, (0x4850 bytes) 2024-04-23T23:50:23.235 INFO:teuthology.orchestra.run.smithi092.stdout:2876 punch from 0x65e4136 to 0x65f0e89, (0xcd53 bytes) 2024-04-23T23:50:23.236 INFO:teuthology.orchestra.run.smithi092.stdout:2877 read 0xc9c97d4 thru 0xc9d8a20 (0xf24d bytes) 2024-04-23T23:50:23.236 INFO:teuthology.orchestra.run.smithi092.stdout:2878 write 0x63e0f31 thru 0x63e2d4c (0x1e1c bytes) 2024-04-23T23:50:23.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:2881 read 0x80941d7 thru 0x809be63 (0x7c8d bytes) 2024-04-23T23:50:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:2882 clone 20 order 24 su 8388608 sc 4 2024-04-23T23:50:23.279 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:23.293+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:23.341 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:23.341 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:23.342 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:23.356+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:23.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:23.443 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:23.458+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:23.493 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:23.493 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:23.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:23.546 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:23.560+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:23.604 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:23.604 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:23.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:23.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:23.654 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:23.668+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:23.746 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:23.760+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:23.758 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:23.759 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:23.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:23.773 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:23.773 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:23.792 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:23.792 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:23.809 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:23.827 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:23.827 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:23.845 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:23.845 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:23.846 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:23.860+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:23.863 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:23.873 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:23.874 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:23.874 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:23.874 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:23.874 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:23.880 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:23.881 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:23.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:23.883 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:23.884 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:23.967 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:23.981+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:24.027 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:3037 3.4 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head [stat,truncate 5242880] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.583539299000002, 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089969830000000001, 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 3037 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456899+0000", 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456900+0000", 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0860000000000001e-06 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456911+0000", 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1467e-05 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456922+0000", 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380999999999999e-05 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.456929+0000", 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8000000000000001e-06 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459092+0000", 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002163697 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459193+0000", 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100709 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459396+0000", 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020303399999999999 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.462879+0000", 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034833899999999998 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465827+0000", 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029476889999999999 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465863+0000", 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5503000000000002e-05 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465896+0000", 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3226999999999999e-05 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6867 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2424832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.73235889, 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077717510000000004, 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6867 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308079+0000", 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308080+0000", 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308082+0000", 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308088+0000", 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308092+0000", 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6140000000000002e-06 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315699+0000", 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076064569999999996 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315761+0000", 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2194999999999997e-05 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315851+0000", 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0109000000000001e-05 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6868 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732339809999999, 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079224340000000008, 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6868 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308098+0000", 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308099+0000", 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4099999999999999e-07 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308101+0000", 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9080000000000002e-06 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308106+0000", 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6869999999999999e-06 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308111+0000", 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315874+0000", 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077625799999999998 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.315936+0000", 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2442999999999998e-05 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316021+0000", 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4592999999999995e-05 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6871 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732321144, 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080708470000000008, 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6871 2024-04-23T23:50:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308117+0000", 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308119+0000", 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8509999999999999e-06 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308128+0000", 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2509999999999994e-06 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308133+0000", 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316044+0000", 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079107259999999999 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316108+0000", 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3937999999999997e-05 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316188+0000", 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0208000000000002e-05 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6872 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732299516000001, 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082303659999999994, 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6872 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308139+0000", 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308142+0000", 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.79e-06 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308147+0000", 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3639999999999998e-06 2024-04-23T23:50:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308151+0000", 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.34e-06 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316212+0000", 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080606099999999993 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316280+0000", 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7692000000000002e-05 2024-04-23T23:50:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316369+0000", 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9148999999999996e-05 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6875 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732280826, 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083900060000000002, 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6875 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308157+0000", 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308158+0000", 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308160+0000", 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9290000000000001e-06 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308166+0000", 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6510000000000004e-06 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308171+0000", 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2640000000000002e-06 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316398+0000", 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082269720000000008 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316466+0000", 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8361999999999994e-05 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316547+0000", 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1311999999999997e-05 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6876 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732261026, 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085708390000000002, 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6876 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308177+0000", 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308178+0000", 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308179+0000", 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.844e-06 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308185+0000", 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.198e-06 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308189+0000", 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1699999999999999e-06 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316574+0000", 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083852639999999999 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316652+0000", 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8078999999999996e-05 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316748+0000", 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5784000000000003e-05 2024-04-23T23:50:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6878 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732241188, 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087314869999999996, 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6878 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308197+0000", 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308199+0000", 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0559999999999999e-06 2024-04-23T23:50:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308205+0000", 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8329999999999999e-06 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308210+0000", 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4519999999999999e-06 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316773+0000", 2024-04-23T23:50:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085637890000000005 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316846+0000", 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2389000000000001e-05 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316928+0000", 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2602000000000006e-05 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6880 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732222796999999, 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089037119999999994, 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6880 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308215+0000", 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308216+0000", 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-04-23T23:50:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308224+0000", 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4409999999999993e-06 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308230+0000", 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8340000000000004e-06 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308234+0000", 2024-04-23T23:50:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4109999999999998e-06 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.316953+0000", 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087181989999999994 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317029+0000", 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6186999999999994e-05 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317119+0000", 2024-04-23T23:50:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0286999999999998e-05 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6883 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732187048, 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090432810000000002, 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6883 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308251+0000", 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308252+0000", 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4199999999999995e-07 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308254+0000", 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4940000000000002e-06 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308262+0000", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3780000000000003e-06 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308267+0000", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8540000000000002e-06 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317145+0000", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088778469999999995 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317212+0000", 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6686000000000005e-05 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317294+0000", 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2379999999999994e-05 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6884 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732164368999999, 2024-04-23T23:50:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092106370000000007, 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6884 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308274+0000", 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2500000000000001e-07 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308276+0000", 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.978e-06 2024-04-23T23:50:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308282+0000", 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308286+0000", 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3649999999999997e-06 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317320+0000", 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009034172 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317394+0000", 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3825000000000006e-05 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317484+0000", 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0340000000000006e-05 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6886 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732146548999999, 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093837590000000002, 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6886 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308292+0000", 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308294+0000", 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.86e-06 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308300+0000", 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308305+0000", 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3249999999999998e-06 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317511+0000", 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092054749999999994 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317587+0000", 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6179000000000004e-05 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317675+0000", 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8595000000000001e-05 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6890 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732127447, 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095370109999999998, 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6890 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308311+0000", 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308313+0000", 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8989999999999999e-06 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308320+0000", 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.827e-06 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308324+0000", 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2830000000000003e-06 2024-04-23T23:50:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317700+0000", 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093762359999999996 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317767+0000", 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6550000000000005e-05 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317848+0000", 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0844e-05 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6892 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732108574000002, 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097323389999999996, 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6892 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308330+0000", 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308332+0000", 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.883e-06 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308338+0000", 2024-04-23T23:50:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9680000000000003e-06 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308342+0000", 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.143e-06 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317873+0000", 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095314449999999995 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.317937+0000", 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3165999999999998e-05 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318062+0000", 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012526900000000001 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6893 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732091056000002, 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099354839999999996, 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6893 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308347+0000", 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308348+0000", 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0399999999999996e-07 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308349+0000", 2024-04-23T23:50:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7540000000000001e-06 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308355+0000", 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3419999999999998e-06 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308359+0000", 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4959999999999998e-06 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318107+0000", 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097478240000000004 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318189+0000", 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1762999999999996e-05 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318283+0000", 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3801000000000005e-05 2024-04-23T23:50:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6894 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732073658000001, 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126606, 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6894 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308364+0000", 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308365+0000", 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308367+0000", 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8920000000000001e-06 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308373+0000", 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8939999999999998e-06 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308377+0000", 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2529999999999998e-06 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318324+0000", 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099474309999999996 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318393+0000", 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8727999999999997e-05 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318491+0000", 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8036999999999997e-05 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6897 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732056038, 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010295080999999999, 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6897 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308382+0000", 2024-04-23T23:50:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308383+0000", 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308384+0000", 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.832e-06 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308396+0000", 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1208e-05 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308400+0000", 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.639e-06 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318519+0000", 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010119025 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318592+0000", 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3124e-05 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318677+0000", 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4792999999999999e-05 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6898 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732031900999999, 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010449478999999999, 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6898 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308406+0000", 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308407+0000", 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308409+0000", 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308414+0000", 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5230000000000003e-06 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308418+0000", 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2039999999999999e-06 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318706+0000", 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01028717 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318774+0000", 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7983000000000003e-05 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318856+0000", 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2047999999999998e-05 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6900 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.732014002, 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010615769000000001, 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6900 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308424+0000", 2024-04-23T23:50:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308427+0000", 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1789999999999998e-06 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308432+0000", 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5990000000000001e-06 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308440+0000", 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318884+0000", 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010443913000000001 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.318953+0000", 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8650000000000002e-05 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319040+0000", 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6908000000000002e-05 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:6902 3.0 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:a [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.731991013999998, 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010766184, 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6902 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308447+0000", 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308455+0000", 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0239999999999997e-06 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308462+0000", 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-06 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.308471+0000", 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4509999999999998e-06 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319063+0000", 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010591685999999999 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319132+0000", 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8628999999999996e-05 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.319213+0000", 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1453999999999996e-05 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.116 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:24.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:24.118 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:24.131+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:24.134 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone19 from 0xdf4e266 (overlap 0x1573d37) to 0x44a5d7 2024-04-23T23:50:24.219 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:24.233+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:24.285 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:24.285 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:24.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:24.339 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:24.353+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:24.421 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:24.435+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:24.490 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:24.491 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:24.491 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:24.491 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:24.491 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:24.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:24.544 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:24.558+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:24.623 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:24.637+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:566 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'3, mlcod=28'1)", 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.773655932, 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001175798, 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915528+0000", 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0580000000000001e-06 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915539+0000", 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638999999999999e-05 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915554+0000", 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5141e-05 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915565+0000", 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0789e-05 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915620+0000", 2024-04-23T23:50:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5189999999999998e-05 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915642+0000", 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2262000000000001e-05 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916672+0000", 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010296109999999999 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916703+0000", 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1108000000000001e-05 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1147 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'4, mlcod=28'2)", 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.757571801000001, 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022957910000000002, 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931628+0000", 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6813999999999999e-05 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931636+0000", 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931644+0000", 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3359999999999997e-06 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931703+0000", 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9598000000000001e-05 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931724+0000", 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0735000000000001e-05 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933865+0000", 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002141165 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933907+0000", 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1768999999999999e-05 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1526 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'16, mlcod=29'14)", 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.303502003999998, 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021410930000000002, 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385682+0000", 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385693+0000", 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0998999999999999e-05 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385710+0000", 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7280000000000001e-05 2024-04-23T23:50:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385719+0000", 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7940000000000008e-06 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385785+0000", 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6109e-05 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385807+0000", 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-05 2024-04-23T23:50:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387778+0000", 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019710769999999999 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387822+0000", 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3665000000000003e-05 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1728 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'20, mlcod=29'18)", 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.073623689000001, 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022012160000000002, 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615560+0000", 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.136e-06 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615580+0000", 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9409e-05 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615597+0000", 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.772e-05 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615606+0000", 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6780000000000006e-06 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615677+0000", 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0714000000000001e-05 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615701+0000", 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4076000000000001e-05 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617717+0000", 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020162729999999999 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617760+0000", 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3210000000000001e-05 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.229620671999999, 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3876 3.9 e33/22 3:93f02760:::rbd_data.1179c62058ea.0000000000000008:head v 33'13, mlcod=33'10)", 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.221469600999999, 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014124350000000001, 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467714+0000", 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1100000000000004e-07 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467723+0000", 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0820000000000005e-06 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467738+0000", 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4598e-05 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467746+0000", 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0620000000000001e-06 2024-04-23T23:50:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467806+0000", 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0591000000000003e-05 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467827+0000", 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0282e-05 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469094+0000", 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001267481 2024-04-23T23:50:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469126+0000", 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1427999999999998e-05 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4487 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'62, mlcod=32'60)", 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.025144983000001, 2024-04-23T23:50:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020224290000000001, 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0599999999999999e-07 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664042+0000", 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.501e-06 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664051+0000", 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6609999999999995e-06 2024-04-23T23:50:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664057+0000", 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6420000000000004e-06 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664109+0000", 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664129+0000", 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0041999999999999e-05 2024-04-23T23:50:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666021+0000", 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00189174 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666060+0000", 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9149000000000001e-05 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4629 3.0 e33/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 33'4, mlcod=31'2)", 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.009707655, 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002138633, 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679476+0000", 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1999999999999998e-07 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679583+0000", 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106983 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679599+0000", 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962999999999999e-05 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679608+0000", 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6659999999999997e-06 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679662+0000", 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4385000000000002e-05 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679685+0000", 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2667000000000002e-05 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681580+0000", 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018954270000000001 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681614+0000", 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3722e-05 2024-04-23T23:50:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5343 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'64, mlcod=33'62)", 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.765709020999999, 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020278729999999999, 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923480+0000", 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923490+0000", 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3919999999999992e-06 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923497+0000", 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923545+0000", 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8647999999999999e-05 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923560+0000", 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5065000000000001e-05 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925471+0000", 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019111340000000001 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925502+0000", 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0321e-05 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.481961712, 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.4 epoch 36/22 rep_tid 124 entries 36'70 (0'0) error 3:25e2e575:::rbd_data.117967c411b0.000000000000002e:head by client.4473.0:6553 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'68)", 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.670117193999999, 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027723230000000001, 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019067+0000", 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9599999999999998e-07 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019079+0000", 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2716e-05 2024-04-23T23:50:24.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019107+0000", 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8232999999999999e-05 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019125+0000", 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717e-05 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019219+0000", 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3455999999999994e-05 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.021838+0000", 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026193050000000002 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 36/22 rep_tid 127 entries 36'23 (0'0) error 3:05311e05:::rbd_data.117967c411b0.0000000000000034:head by client.4473.0:6561 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'21)", 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.666727898000001, 2024-04-23T23:50:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002186155, 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022456+0000", 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7899999999999998e-07 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022464+0000", 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022484+0000", 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0369000000000001e-05 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022495+0000", 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1205e-05 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022559+0000", 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3465000000000003e-05 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.024641+0000", 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020822810000000001 2024-04-23T23:50:24.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6580 3.0 e36/22 3:053657a9:::rbd_data.117967c411b0.000000000000003a:head v 36'26, mlcod=36'24)", 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.657865371, 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011394210000000001, 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031318+0000", 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-07 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031324+0000", 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031338+0000", 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4036000000000001e-05 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031346+0000", 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2450000000000001e-06 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031392+0000", 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6214000000000002e-05 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031410+0000", 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7859e-05 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032426+0000", 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010161530000000001 2024-04-23T23:50:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032457+0000", 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1151e-05 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6656 3.4 e36/22 3:23b8cbcf:::rbd_data.117967c411b0.000000000000001f:head v 36'72, mlcod=36'70)", 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.640611609, 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001073517, 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048571+0000", 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048572+0000", 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048579+0000", 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048593+0000", 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4331000000000001e-05 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048601+0000", 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8539999999999997e-06 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048665+0000", 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3748e-05 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.048684+0000", 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9088000000000001e-05 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049604+0000", 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091966300000000003 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049645+0000", 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0936999999999998e-05 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.640092407000001, 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:24.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:20374 3.0 e40/22 3:01cd6562:::rbd_data.11793b798a9c.000000000000000e:head v 40'50, mlcod=38'48)", 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.702178118999999, 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001039091, 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987005+0000", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987012+0000", 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9179999999999997e-06 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987022+0000", 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8320000000000001e-06 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987030+0000", 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987092+0000", 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2902000000000002e-05 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.987118+0000", 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5624000000000001e-05 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.988004+0000", 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088584899999999997 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:10.988044+0000", 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9869000000000001e-05 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:24539 3.9 e42/22 3:908cbee1:::rbd_data.1179cfb6b7da.0000000000000031:head v 42'202, mlcod=42'200)", 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:14.715977+0000", 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9732060709999999, 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013857749999999999, 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715977+0000", 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715977+0000", 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715978+0000", 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1289999999999999e-06 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715989+0000", 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1306e-05 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716006+0000", 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7110999999999999e-05 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716015+0000", 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7590000000000003e-06 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716447+0000", 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043230100000000001 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716468+0000", 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0873000000000002e-05 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.717319+0000", 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085117600000000004 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.717363+0000", 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3120000000000001e-05 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:24639 3.9 e42/22 3:908cbee1:::rbd_data.1179cfb6b7da.0000000000000031:head v 42'204, mlcod=42'202)", 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:14.826550+0000", 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8626330099999997, 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011463809999999999, 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826550+0000", 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826550+0000", 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826552+0000", 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2170000000000001e-06 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826568+0000", 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5855999999999999e-05 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826600+0000", 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1742999999999997e-05 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826617+0000", 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7351999999999999e-05 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826761+0000", 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014441599999999999 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826822+0000", 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0732000000000001e-05 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.827657+0000", 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083490400000000003 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.827696+0000", 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9161e-05 2024-04-23T23:50:24.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:24916 3.9 e43/22 3:94620a61:::rbd_data.1179cfb6b7da.000000000000002f:head v 43'209, mlcod=42'206)", 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6341421710000006, 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011846070000000001, 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6899999999999997e-07 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055048+0000", 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6309999999999997e-06 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055057+0000", 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4609999999999997e-06 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055064+0000", 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1500000000000002e-06 2024-04-23T23:50:24.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055289+0000", 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000224868 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055312+0000", 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2949e-05 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.056192+0000", 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087976099999999995 2024-04-23T23:50:24.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.056225+0000", 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3217999999999999e-05 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:31114 3.4 e47/22 3:25439e5d:::rbd_data.11797b718ff4.000000000000000a:head v 47'82, mlcod=44'80)", 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:21.583198+0000", 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.105984683, 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001416004, 2024-04-23T23:50:24.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583198+0000", 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583198+0000", 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583199+0000", 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.251e-06 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583208+0000", 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9190000000000005e-06 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583229+0000", 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0611e-05 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583239+0000", 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6560000000000004e-06 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583297+0000", 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8721000000000002e-05 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583319+0000", 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1569e-05 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.584572+0000", 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001253299 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.584614+0000", 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1977999999999997e-05 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:24.761 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:24.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:24.763 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-04-23T23:50:24.764 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:24.776+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:24.777 INFO:teuthology.orchestra.run.smithi092.stdout:2883 punch from 0x2d1793f to 0x2d2030b, (0x89cc bytes) 2024-04-23T23:50:24.781 INFO:teuthology.orchestra.run.smithi092.stdout:2884 read 0xd93ca9c thru 0xd9472d2 (0xa837 bytes) 2024-04-23T23:50:24.781 INFO:teuthology.orchestra.run.smithi092.stdout:2885 read 0x98e1481 thru 0x98e2cf4 (0x1874 bytes) 2024-04-23T23:50:24.781 INFO:teuthology.orchestra.run.smithi092.stdout:2886 write 0x2ebb33f thru 0x2ec32fd (0x7fbf bytes) 2024-04-23T23:50:24.864 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:24.878+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:24.914 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:24.914 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:24.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.914 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44195 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.914 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.923628+0000", 2024-04-23T23:50:24.914 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.002040222, 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020642350000000002, 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44195 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923628+0000", 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923628+0000", 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923628+0000", 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.23e-07 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923633+0000", 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2409999999999997e-06 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923643+0000", 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0669000000000001e-05 2024-04-23T23:50:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923656+0000", 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2853999999999999e-05 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44197 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.923751+0000", 2024-04-23T23:50:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0019169910000000001, 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020066189999999999, 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44197 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923751+0000", 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923751+0000", 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923752+0000", 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1299999999999998e-07 2024-04-23T23:50:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923756+0000", 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5390000000000001e-06 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923768+0000", 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1328000000000001e-05 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923776+0000", 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8849999999999999e-06 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44199 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.923912+0000", 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001756241, 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001904053, 2024-04-23T23:50:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44199 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923912+0000", 2024-04-23T23:50:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923912+0000", 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923912+0000", 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2699999999999998e-07 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923917+0000", 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.194e-06 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923927+0000", 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0380000000000001e-05 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.923934+0000", 2024-04-23T23:50:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1799999999999999e-06 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44201 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924197+0000", 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0014710190000000001, 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016802189999999999, 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44201 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924197+0000", 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924197+0000", 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924198+0000", 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4300000000000002e-07 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924202+0000", 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0350000000000003e-06 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924212+0000", 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0648999999999999e-05 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924220+0000", 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:50:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44203 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924406+0000", 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001262095, 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001526634, 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44203 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924406+0000", 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924406+0000", 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924407+0000", 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924411+0000", 2024-04-23T23:50:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3839999999999999e-06 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924422+0000", 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0845e-05 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924429+0000", 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1269999999999997e-06 2024-04-23T23:50:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44205 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924455+0000", 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0012129420000000001, 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001534103, 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44205 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924455+0000", 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924455+0000", 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924456+0000", 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3799999999999997e-07 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924458+0000", 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.723e-06 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924467+0000", 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3280000000000006e-06 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924473+0000", 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2740000000000004e-06 2024-04-23T23:50:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44207 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924484+0000", 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0011838980000000001, 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001561207, 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44207 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924484+0000", 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924484+0000", 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924485+0000", 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5599999999999995e-07 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924488+0000", 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0970000000000002e-06 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924499+0000", 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1263e-05 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924505+0000", 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2129999999999996e-06 2024-04-23T23:50:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44209 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924536+0000", 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001131953, 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001563643, 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44209 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924536+0000", 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924536+0000", 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924537+0000", 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.06e-07 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924540+0000", 2024-04-23T23:50:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0340000000000001e-06 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924548+0000", 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2090000000000005e-06 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924554+0000", 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44211 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924629+0000", 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0010386989999999999, 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001524608, 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44211 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924629+0000", 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924629+0000", 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924630+0000", 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8100000000000002e-07 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924633+0000", 2024-04-23T23:50:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4659999999999999e-06 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924643+0000", 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0729999999999994e-06 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924649+0000", 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4880000000000004e-06 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44214 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924783+0000", 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00088465099999999997, 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001426692, 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44214 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924783+0000", 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924783+0000", 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924784+0000", 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1500000000000003e-07 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924788+0000", 2024-04-23T23:50:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924798+0000", 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0301e-05 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924805+0000", 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0389999999999999e-06 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44215 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924836+0000", 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.000832003, 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001428143, 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44215 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924836+0000", 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924836+0000", 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924837+0000", 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1900000000000001e-07 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924840+0000", 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924848+0000", 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924854+0000", 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44217 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.924975+0000", 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00069335499999999999, 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343459, 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44217 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924975+0000", 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924975+0000", 2024-04-23T23:50:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924975+0000", 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8599999999999998e-07 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924979+0000", 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4589999999999998e-06 2024-04-23T23:50:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924988+0000", 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4760000000000004e-06 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.924995+0000", 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000001e-06 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44219 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925126+0000", 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00054157700000000001, 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012457639999999999, 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44219 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925126+0000", 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925126+0000", 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925127+0000", 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6300000000000004e-07 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925132+0000", 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3930000000000001e-06 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925143+0000", 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1017e-05 2024-04-23T23:50:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925150+0000", 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8110000000000003e-06 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44221 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925169+0000", 2024-04-23T23:50:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00049938200000000004, 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012572810000000001, 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44221 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925169+0000", 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925169+0000", 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925169+0000", 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3300000000000002e-07 2024-04-23T23:50:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925171+0000", 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.339e-06 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925180+0000", 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.157e-06 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925185+0000", 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44223 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925309+0000", 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.000358563, 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001170602, 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44223 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925309+0000", 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925309+0000", 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925310+0000", 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4499999999999996e-07 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925314+0000", 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9770000000000002e-06 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925323+0000", 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9139999999999997e-06 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925329+0000", 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9850000000000003e-06 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44225 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925449+0000", 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00021840100000000001, 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010848500000000001, 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44225 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925449+0000", 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925449+0000", 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925450+0000", 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0400000000000005e-07 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925454+0000", 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0219999999999998e-06 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925466+0000", 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1255e-05 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925473+0000", 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1010000000000003e-06 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44227 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925600+0000", 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.7545000000000005e-05, 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098812899999999992, 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44227 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925600+0000", 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925600+0000", 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925601+0000", 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4e-07 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925605+0000", 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0010000000000003e-06 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925614+0000", 2024-04-23T23:50:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925626+0000", 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2371000000000001e-05 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44229 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925682+0000", 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9999862, 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096657799999999995, 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44229 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925682+0000", 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925682+0000", 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925683+0000", 2024-04-23T23:50:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.99e-07 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925686+0000", 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2899999999999998e-06 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925693+0000", 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3189999999999997e-06 2024-04-23T23:50:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925699+0000", 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9789999999999998e-06 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44231 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.925843+0000", 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9998255, 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086153300000000005, 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44231 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925843+0000", 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925843+0000", 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925843+0000", 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0999999999999997e-07 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925848+0000", 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925859+0000", 2024-04-23T23:50:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1231e-05 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.925868+0000", 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2350000000000005e-06 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44233 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926051+0000", 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9996171, 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00070933200000000002, 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44233 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926051+0000", 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926051+0000", 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926051+0000", 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4900000000000005e-07 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926055+0000", 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926064+0000", 2024-04-23T23:50:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926071+0000", 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6660000000000002e-06 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44235 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926316+0000", 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9993515, 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00049993900000000003, 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44235 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926316+0000", 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926316+0000", 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926317+0000", 2024-04-23T23:50:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0200000000000001e-07 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926321+0000", 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.715e-06 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926329+0000", 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0530000000000007e-06 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926334+0000", 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.75e-06 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44237 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926588+0000", 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9990792, 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000287108, 2024-04-23T23:50:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44237 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926588+0000", 2024-04-23T23:50:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926588+0000", 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926589+0000", 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6400000000000002e-07 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926593+0000", 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6389999999999998e-06 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926600+0000", 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0219999999999999e-06 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926606+0000", 2024-04-23T23:50:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8980000000000001e-06 2024-04-23T23:50:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44239 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926635+0000", 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9990335, 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029466300000000002, 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44239 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926635+0000", 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926635+0000", 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926635+0000", 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6899999999999997e-07 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926638+0000", 2024-04-23T23:50:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4360000000000001e-06 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926644+0000", 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2920000000000001e-06 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926647+0000", 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44240 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926656+0000", 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9990115, 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000327063, 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44240 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926656+0000", 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926656+0000", 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926657+0000", 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3999999999999997e-07 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926658+0000", 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.46e-06 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926663+0000", 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4719999999999997e-06 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926667+0000", 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9960000000000004e-06 2024-04-23T23:50:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44242 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926671+0000", 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9989972, 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036782900000000002, 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44242 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926671+0000", 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926671+0000", 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926671+0000", 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5600000000000001e-07 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926672+0000", 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2729999999999999e-06 2024-04-23T23:50:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926676+0000", 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2370000000000001e-06 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926678+0000", 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5320000000000001e-06 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44244 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926681+0000", 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9989867, 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00041375800000000002, 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44244 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926681+0000", 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926681+0000", 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926681+0000", 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926683+0000", 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1379999999999999e-06 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926685+0000", 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8569999999999999e-06 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926688+0000", 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4820000000000001e-06 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44246 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926707+0000", 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9989605, 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044420000000000001, 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44246 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926707+0000", 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926707+0000", 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926708+0000", 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7100000000000002e-07 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926710+0000", 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5900000000000002e-06 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926718+0000", 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.131e-06 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926724+0000", 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6359999999999997e-06 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44248 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.926950+0000", 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9987178, 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025771300000000002, 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44248 2024-04-23T23:50:25.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926950+0000", 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926950+0000", 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926951+0000", 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1500000000000003e-07 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926955+0000", 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1500000000000001e-06 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926966+0000", 2024-04-23T23:50:25.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1355999999999999e-05 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.926973+0000", 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8480000000000003e-06 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44250 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.024 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.927016+0000", 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.998652, 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024708999999999999, 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44250 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927016+0000", 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927016+0000", 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927017+0000", 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.61e-07 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927023+0000", 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.84e-06 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927038+0000", 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4891e-05 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927044+0000", 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5520000000000001e-06 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44252 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.927152+0000", 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9985156, 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000166267, 2024-04-23T23:50:25.028 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44252 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927152+0000", 2024-04-23T23:50:25.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927152+0000", 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927153+0000", 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6899999999999997e-07 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927157+0000", 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927166+0000", 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6290000000000005e-06 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927172+0000", 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0249999999999999e-06 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44254 3.14 3.debd3b74 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.927276+0000", 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4294967295.9983916, 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8596000000000003e-05, 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44254 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927276+0000", 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927276+0000", 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927277+0000", 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.13e-07 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927280+0000", 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.078e-06 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927290+0000", 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7059999999999999e-06 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.927296+0000", 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7649999999999999e-06 2024-04-23T23:50:25.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.036 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-23T23:50:25.036 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 31 2024-04-23T23:50:25.036 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:25.037 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:25.051+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:25.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:25.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:50:25.138 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:25.152+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:25.142 INFO:teuthology.orchestra.run.smithi092.stdout:2887 trunc from 0xdf4e266 to 0x7b86f8d 2024-04-23T23:50:25.152 INFO:teuthology.orchestra.run.smithi092.stdout:2889 read 0x2664c31 thru 0x2667a6d (0x2e3d bytes) 2024-04-23T23:50:25.157 INFO:teuthology.orchestra.run.smithi092.stdout:2891 trunc from 0x7b86f8d to 0xb45a63d 2024-04-23T23:50:25.162 INFO:teuthology.orchestra.run.smithi092.stdout:2892 write 0x108bfdb thru 0x1097f43 (0xbf69 bytes) 2024-04-23T23:50:25.195 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:25.195 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:25.195 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:25.195 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:25.195 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:25.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:25.240 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:25.254+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:25.341 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:25.355+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.93517666499999996, 2024-04-23T23:50:25.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:50:25.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:50:25.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.93516678499999994, 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:50:25.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:50:25.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:25.363 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.93495643799999995, 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:50:25.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:50:25.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.93312509899999996, 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:50:25.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:50:25.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.93105546800000005, 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.376 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41896523200000002, 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:50:25.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:50:25.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:50:25.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41893957599999998, 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:50:25.381 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:50:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:50:25.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41887208599999998, 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.386 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:50:25.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:50:25.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:50:25.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41882702500000002, 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.390 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:50:25.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:50:25.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41879052500000002, 2024-04-23T23:50:25.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41877356599999999, 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:50:25.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41869488199999999, 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:25.407 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.418627849, 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:25.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.411 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41849424800000001, 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.412 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41807556800000001, 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:25.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:25.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41792492599999997, 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:25.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41779954400000002, 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:50:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41758402700000002, 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:50:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.41734669299999999, 2024-04-23T23:50:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:50:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:50:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.417106798, 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:50:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:50:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:50:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:50:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.442 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:25.443 INFO:teuthology.orchestra.run.smithi092.stdout:2893 trunc from 0xb45a63d to 0x936a002 2024-04-23T23:50:25.443 INFO:teuthology.orchestra.run.smithi092.stdout:2894 read 0x3da2a6 thru 0x3dcff5 (0x2d50 bytes) 2024-04-23T23:50:25.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:25.444 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:25.457+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:25.451 INFO:teuthology.orchestra.run.smithi092.stdout:2895 write 0xe57097f thru 0xe57ec05 (0xe287 bytes) 2024-04-23T23:50:25.469 INFO:teuthology.orchestra.run.smithi092.stdout:2896 trunc from 0xe57ec06 to 0x1649408 2024-04-23T23:50:25.544 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:25.558+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:25.645 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:25.659+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:25.651 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51034 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644499+0000", 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0082568840000000008, 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082719709999999995, 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51034 2024-04-23T23:50:25.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644499+0000", 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644499+0000", 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644500+0000", 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644503+0000", 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2320000000000001e-06 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644506+0000", 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.484e-06 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644510+0000", 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799999999999998e-06 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51066 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645177+0000", 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0075788110000000004, 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007628775, 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51066 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645177+0000", 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645177+0000", 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645178+0000", 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2599999999999998e-07 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645179+0000", 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.409e-06 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645183+0000", 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4570000000000001e-06 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645186+0000", 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0170000000000001e-06 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51098 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645774+0000", 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.006982205, 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070692799999999998, 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51098 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645774+0000", 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645774+0000", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645774+0000", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6100000000000002e-07 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645776+0000", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4190000000000001e-06 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645780+0000", 2024-04-23T23:50:25.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8510000000000001e-06 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645782+0000", 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.762e-06 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51130 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646618+0000", 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0061378140000000001, 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062524989999999999, 2024-04-23T23:50:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51130 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646618+0000", 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646618+0000", 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646619+0000", 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6400000000000003e-07 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646621+0000", 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.765e-06 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646626+0000", 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5709999999999997e-06 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646631+0000", 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3699999999999997e-06 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51162 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647358+0000", 2024-04-23T23:50:25.664 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0053986709999999999, 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055400800000000002, 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51162 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647358+0000", 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647358+0000", 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647358+0000", 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9500000000000003e-07 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647360+0000", 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.995e-06 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647366+0000", 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9989999999999997e-06 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647372+0000", 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0249999999999999e-06 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51194 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648094+0000", 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0046620840000000004, 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048334440000000001, 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51194 2024-04-23T23:50:25.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648094+0000", 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648094+0000", 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648095+0000", 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9100000000000004e-07 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648098+0000", 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2389999999999998e-06 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648102+0000", 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3540000000000002e-06 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648107+0000", 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51226 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648891+0000", 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0038654940000000001, 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040631349999999998, 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51226 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648891+0000", 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648891+0000", 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648891+0000", 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7599999999999998e-07 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648893+0000", 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648897+0000", 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5650000000000003e-06 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648909+0000", 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1545999999999999e-05 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51035 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644514+0000", 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0082425239999999993, 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084680680000000005, 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51035 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644514+0000", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644514+0000", 2024-04-23T23:50:25.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644514+0000", 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644515+0000", 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.068e-06 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644518+0000", 2024-04-23T23:50:25.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0000000000000001e-06 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644521+0000", 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6850000000000001e-06 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51067 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645218+0000", 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00753808, 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077904480000000002, 2024-04-23T23:50:25.677 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51067 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645218+0000", 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645218+0000", 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645219+0000", 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-07 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645220+0000", 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4079999999999999e-06 2024-04-23T23:50:25.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645224+0000", 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.534e-06 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645227+0000", 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4999999999999999e-06 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.680 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51099 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645805+0000", 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0069508139999999996, 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072313969999999997, 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51099 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645805+0000", 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645805+0000", 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645806+0000", 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3799999999999997e-07 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645808+0000", 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.857e-06 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645813+0000", 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8550000000000001e-06 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645816+0000", 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.853e-06 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51131 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646636+0000", 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0061206209999999997, 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064277300000000004, 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51131 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646636+0000", 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646636+0000", 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646636+0000", 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4200000000000002e-07 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646638+0000", 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.708e-06 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646643+0000", 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4879999999999998e-06 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646649+0000", 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.361e-06 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51163 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647396+0000", 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0053605609999999998, 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056941179999999997, 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51163 2024-04-23T23:50:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647396+0000", 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647396+0000", 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647396+0000", 2024-04-23T23:50:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6100000000000002e-07 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647398+0000", 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6810000000000001e-06 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647401+0000", 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8319999999999999e-06 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647405+0000", 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0539999999999999e-06 2024-04-23T23:50:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51195 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648118+0000", 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0046383969999999998, 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049983670000000001, 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51195 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648118+0000", 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648118+0000", 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648118+0000", 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9999999999999998e-07 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648120+0000", 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0329999999999998e-06 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648127+0000", 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2550000000000003e-06 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648131+0000", 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7520000000000004e-06 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51227 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648916+0000", 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0038397919999999999, 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042262180000000003, 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51227 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648916+0000", 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648916+0000", 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648917+0000", 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3599999999999999e-07 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648919+0000", 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5440000000000001e-06 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648923+0000", 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.777e-06 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648926+0000", 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1889999999999999e-06 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51036 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644524+0000", 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0082317910000000005, 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086453769999999992, 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51036 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644524+0000", 2024-04-23T23:50:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644524+0000", 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644525+0000", 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644526+0000", 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0100000000000001e-06 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644529+0000", 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0230000000000001e-06 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644531+0000", 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5359999999999999e-06 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51068 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645230+0000", 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0075259690000000004, 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079659680000000004, 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51068 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645230+0000", 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645230+0000", 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645231+0000", 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.22e-07 2024-04-23T23:50:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645232+0000", 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0270000000000001e-06 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645235+0000", 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0110000000000001e-06 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645238+0000", 2024-04-23T23:50:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3890000000000001e-06 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51100 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645819+0000", 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0069369770000000004, 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074034610000000001, 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51100 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645819+0000", 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645819+0000", 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645820+0000", 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8000000000000001e-07 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645821+0000", 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1119999999999999e-06 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645832+0000", 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0753e-05 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645836+0000", 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5560000000000001e-06 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51132 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646660+0000", 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.006096125, 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00659328, 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51132 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646660+0000", 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646660+0000", 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646660+0000", 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6600000000000003e-07 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646662+0000", 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.655e-06 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646666+0000", 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7189999999999999e-06 2024-04-23T23:50:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646669+0000", 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51164 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647408+0000", 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0053479720000000003, 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058715089999999996, 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51164 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647408+0000", 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647408+0000", 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647409+0000", 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2300000000000002e-07 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647410+0000", 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5290000000000001e-06 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647415+0000", 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5220000000000001e-06 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647417+0000", 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.638e-06 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51196 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648153+0000", 2024-04-23T23:50:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0046028689999999999, 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051529109999999996, 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51196 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648153+0000", 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648153+0000", 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648154+0000", 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4999999999999999e-07 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648155+0000", 2024-04-23T23:50:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6190000000000001e-06 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648159+0000", 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8060000000000001e-06 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648162+0000", 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1080000000000001e-06 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51228 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648930+0000", 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0038265040000000001, 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044027780000000004, 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51228 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648930+0000", 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648930+0000", 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648930+0000", 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648931+0000", 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0759999999999999e-06 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648934+0000", 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.157e-06 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648937+0000", 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.65e-06 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51037 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644535+0000", 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0082212829999999994, 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088246820000000004, 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51037 2024-04-23T23:50:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644535+0000", 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644535+0000", 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644535+0000", 2024-04-23T23:50:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644536+0000", 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.02e-06 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644540+0000", 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3569999999999998e-06 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644543+0000", 2024-04-23T23:50:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5640000000000001e-06 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51069 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645242+0000", 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.007514728, 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081461829999999995, 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51069 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645242+0000", 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645242+0000", 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645242+0000", 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.39e-07 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645243+0000", 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.057e-06 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645246+0000", 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1939999999999998e-06 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645249+0000", 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7180000000000001e-06 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51101 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645854+0000", 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0069024739999999996, 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075601619999999996, 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51101 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645854+0000", 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645854+0000", 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645854+0000", 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6899999999999998e-07 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645856+0000", 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6929999999999999e-06 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645860+0000", 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7270000000000001e-06 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645862+0000", 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7709999999999998e-06 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51133 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646672+0000", 2024-04-23T23:50:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0060845500000000002, 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067687340000000002, 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51133 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646672+0000", 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646672+0000", 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646672+0000", 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.11e-07 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646673+0000", 2024-04-23T23:50:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.257e-06 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646678+0000", 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1399999999999999e-06 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646681+0000", 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7209999999999999e-06 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51165 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647436+0000", 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0053203369999999996, 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060307349999999997, 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51165 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647436+0000", 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647436+0000", 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647436+0000", 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.37e-07 2024-04-23T23:50:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647445+0000", 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2009999999999999e-06 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647453+0000", 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7080000000000003e-06 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647459+0000", 2024-04-23T23:50:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2820000000000004e-06 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51197 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648166+0000", 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0045904240000000001, 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053272930000000003, 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51197 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648166+0000", 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648166+0000", 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648166+0000", 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3700000000000001e-07 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648168+0000", 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6390000000000001e-06 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648172+0000", 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6399999999999996e-06 2024-04-23T23:50:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648176+0000", 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1489999999999998e-06 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51229 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648940+0000", 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0038165460000000001, 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045805469999999999, 2024-04-23T23:50:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51229 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648940+0000", 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648940+0000", 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648940+0000", 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1199999999999999e-07 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648941+0000", 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4600000000000003e-07 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648945+0000", 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648948+0000", 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8399999999999999e-06 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51038 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644546+0000", 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0082101940000000005, 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090012449999999997, 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51038 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644546+0000", 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644546+0000", 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644546+0000", 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.28e-07 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644547+0000", 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5900000000000005e-07 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644550+0000", 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9160000000000001e-06 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644553+0000", 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.576e-06 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51070 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645252+0000", 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0075044999999999999, 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083218779999999996, 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51070 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645252+0000", 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645252+0000", 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645252+0000", 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.23e-07 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645257+0000", 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2639999999999999e-06 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645261+0000", 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3129999999999999e-06 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645264+0000", 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3560000000000001e-06 2024-04-23T23:50:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51102 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645895+0000", 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0068616750000000002, 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077055120000000003, 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51102 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645895+0000", 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645895+0000", 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645895+0000", 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2e-07 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645896+0000", 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4020000000000001e-06 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645900+0000", 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5379999999999998e-06 2024-04-23T23:50:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645903+0000", 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7879999999999998e-06 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51134 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646695+0000", 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0060612110000000004, 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312940000000002, 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51134 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646695+0000", 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646695+0000", 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646696+0000", 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9100000000000004e-07 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646698+0000", 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9740000000000001e-06 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646704+0000", 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0510000000000002e-06 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646709+0000", 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3619999999999997e-06 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51166 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647476+0000", 2024-04-23T23:50:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0052803679999999997, 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061768300000000003, 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51166 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647476+0000", 2024-04-23T23:50:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647476+0000", 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647476+0000", 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.58e-07 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647478+0000", 2024-04-23T23:50:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.796e-06 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647482+0000", 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9450000000000003e-06 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647485+0000", 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51198 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648188+0000", 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.004567977, 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054905600000000002, 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51198 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648188+0000", 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648188+0000", 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648189+0000", 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648191+0000", 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0509999999999999e-06 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648197+0000", 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0329999999999997e-06 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648201+0000", 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3749999999999996e-06 2024-04-23T23:50:25.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51230 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648951+0000", 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0038055950000000002, 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047622610000000003, 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51230 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648951+0000", 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648951+0000", 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648951+0000", 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648952+0000", 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9900000000000009e-07 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648955+0000", 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0419999999999998e-06 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648958+0000", 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.349e-06 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51039 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644575+0000", 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0081813919999999991, 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091656640000000004, 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51039 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644575+0000", 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644575+0000", 2024-04-23T23:50:25.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644575+0000", 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5499999999999999e-07 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644577+0000", 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644580+0000", 2024-04-23T23:50:25.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4620000000000001e-06 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644583+0000", 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8990000000000001e-06 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51071 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645268+0000", 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074885409999999996, 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084993659999999995, 2024-04-23T23:50:25.767 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51071 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645268+0000", 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645268+0000", 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645268+0000", 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645269+0000", 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0699999999999999e-06 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645272+0000", 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645275+0000", 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7489999999999999e-06 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51103 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.770 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645906+0000", 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.006850376, 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078874849999999996, 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51103 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645906+0000", 2024-04-23T23:50:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645906+0000", 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645906+0000", 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1199999999999999e-07 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645907+0000", 2024-04-23T23:50:25.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0249999999999999e-06 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645910+0000", 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0139999999999999e-06 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645913+0000", 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7580000000000001e-06 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.773 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51135 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646731+0000", 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0060248130000000004, 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070880810000000004, 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51135 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646731+0000", 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646731+0000", 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646732+0000", 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5199999999999998e-07 2024-04-23T23:50:25.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646733+0000", 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.719e-06 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646737+0000", 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9060000000000004e-06 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646740+0000", 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9639999999999999e-06 2024-04-23T23:50:25.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51167 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647488+0000", 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0052681000000000004, 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063632309999999996, 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.777 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51167 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647488+0000", 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647488+0000", 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647488+0000", 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647490+0000", 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6470000000000001e-06 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647495+0000", 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8369999999999996e-06 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647502+0000", 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7089999999999996e-06 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51199 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648220+0000", 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0045359620000000002, 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056590729999999997, 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51199 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648220+0000", 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648220+0000", 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648221+0000", 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5100000000000001e-07 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648222+0000", 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3710000000000001e-06 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648226+0000", 2024-04-23T23:50:25.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5029999999999999e-06 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648231+0000", 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4780000000000002e-06 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51231 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648961+0000", 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0037952149999999998, 2024-04-23T23:50:25.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049450279999999998, 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51231 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648961+0000", 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648961+0000", 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648961+0000", 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8599999999999999e-07 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648962+0000", 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2999999999999999e-07 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648965+0000", 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648968+0000", 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51040 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644586+0000", 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0081700079999999994, 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093480869999999997, 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51040 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644586+0000", 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644586+0000", 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644587+0000", 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644588+0000", 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.877e-06 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644592+0000", 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1599999999999998e-06 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644594+0000", 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6539999999999999e-06 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51072 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645278+0000", 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074783699999999998, 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086836169999999994, 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51072 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.790 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645278+0000", 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645278+0000", 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645278+0000", 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6300000000000001e-07 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645279+0000", 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645282+0000", 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.033e-06 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645285+0000", 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6690000000000002e-06 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51104 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645916+0000", 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0068402719999999997, 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080724230000000004, 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.793 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51104 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645916+0000", 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645916+0000", 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645916+0000", 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7e-07 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645917+0000", 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7300000000000004e-07 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645920+0000", 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9059999999999998e-06 2024-04-23T23:50:25.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645924+0000", 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5609999999999999e-06 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51136 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646745+0000", 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0060113079999999999, 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007270108, 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51136 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646745+0000", 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646745+0000", 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646746+0000", 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3149999999999999e-06 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646748+0000", 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6500000000000001e-06 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646753+0000", 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5249999999999996e-06 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646758+0000", 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5070000000000002e-06 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51168 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647512+0000", 2024-04-23T23:50:25.799 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.005244683, 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065299829999999996, 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51168 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647512+0000", 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647512+0000", 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647512+0000", 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9800000000000004e-07 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647514+0000", 2024-04-23T23:50:25.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0609999999999998e-06 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647525+0000", 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0922e-05 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647530+0000", 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51200 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648248+0000", 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0045085409999999996, 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005823206, 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51200 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648248+0000", 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648248+0000", 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648248+0000", 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8299999999999997e-07 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648250+0000", 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.012e-06 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648256+0000", 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.993e-06 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648261+0000", 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5539999999999999e-06 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.805 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51232 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648971+0000", 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.003785489, 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051266929999999999, 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.806 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51232 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648971+0000", 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648971+0000", 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648971+0000", 2024-04-23T23:50:25.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1199999999999999e-07 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648972+0000", 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648976+0000", 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5760000000000002e-06 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648978+0000", 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7010000000000001e-06 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51041 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644610+0000", 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0081463019999999994, 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095149959999999995, 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51041 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644610+0000", 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644610+0000", 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644610+0000", 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.27e-07 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644612+0000", 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3480000000000001e-06 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644615+0000", 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4829999999999999e-06 2024-04-23T23:50:25.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644619+0000", 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.737e-06 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51073 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645288+0000", 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074684909999999998, 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088638659999999998, 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.812 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51073 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645288+0000", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645288+0000", 2024-04-23T23:50:25.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645288+0000", 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645290+0000", 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8640000000000001e-06 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645293+0000", 2024-04-23T23:50:25.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645296+0000", 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2370000000000001e-06 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51105 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645939+0000", 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0068169529999999997, 2024-04-23T23:50:25.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082402350000000003, 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51105 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645939+0000", 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645939+0000", 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645940+0000", 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2500000000000001e-07 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645941+0000", 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.48e-06 2024-04-23T23:50:25.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645945+0000", 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4819999999999999e-06 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645947+0000", 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.723e-06 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51137 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646768+0000", 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0059883740000000003, 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074380699999999998, 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51137 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646768+0000", 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646768+0000", 2024-04-23T23:50:25.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646768+0000", 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2199999999999999e-07 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646770+0000", 2024-04-23T23:50:25.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8080000000000001e-06 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646776+0000", 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3249999999999996e-06 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646781+0000", 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3549999999999998e-06 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51169 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647536+0000", 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0052202400000000001, 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006696474, 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51169 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647536+0000", 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647536+0000", 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647536+0000", 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647538+0000", 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.658e-06 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647542+0000", 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.313e-06 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647546+0000", 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3189999999999999e-06 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51201 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648266+0000", 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0044899220000000004, 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059927829999999998, 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51201 2024-04-23T23:50:25.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648266+0000", 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648266+0000", 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648267+0000", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6300000000000001e-07 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648272+0000", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.95e-06 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648278+0000", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3409999999999999e-06 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648281+0000", 2024-04-23T23:50:25.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2600000000000001e-06 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51233 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648981+0000", 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0037752390000000001, 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053051870000000003, 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.829 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51233 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648981+0000", 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648981+0000", 2024-04-23T23:50:25.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649108+0000", 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000126747 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649110+0000", 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7329999999999999e-06 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649130+0000", 2024-04-23T23:50:25.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9618000000000001e-05 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649136+0000", 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7509999999999997e-06 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51042 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644622+0000", 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0081338130000000002, 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096916650000000003, 2024-04-23T23:50:25.832 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51042 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644622+0000", 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644622+0000", 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644623+0000", 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644624+0000", 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0100000000000001e-06 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644627+0000", 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1020000000000001e-06 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644630+0000", 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4589999999999998e-06 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51074 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.835 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645299+0000", 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074571120000000001, 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090419220000000008, 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51074 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645299+0000", 2024-04-23T23:50:25.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645299+0000", 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645299+0000", 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.34e-07 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645300+0000", 2024-04-23T23:50:25.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.47e-07 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645303+0000", 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0469999999999998e-06 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645306+0000", 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6850000000000001e-06 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51106 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645984+0000", 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0067727339999999999, 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083842929999999993, 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51106 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645984+0000", 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645984+0000", 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645984+0000", 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.96e-07 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645986+0000", 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3719999999999999e-06 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645990+0000", 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9060000000000004e-06 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645993+0000", 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8849999999999999e-06 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51138 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646796+0000", 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0059602630000000004, 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075993240000000002, 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51138 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.842 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646796+0000", 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646796+0000", 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646796+0000", 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7799999999999997e-07 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646798+0000", 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.587e-06 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646802+0000", 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6990000000000001e-06 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646804+0000", 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9229999999999998e-06 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51170 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647549+0000", 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0052071959999999999, 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068729359999999996, 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.845 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51170 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647549+0000", 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647549+0000", 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647549+0000", 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3300000000000001e-07 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647550+0000", 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8700000000000004e-07 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647553+0000", 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.174e-06 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647556+0000", 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6809999999999998e-06 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51202 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648291+0000", 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0044647979999999999, 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061570720000000004, 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51202 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648291+0000", 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648291+0000", 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648292+0000", 2024-04-23T23:50:25.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8700000000000002e-07 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648293+0000", 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6309999999999999e-06 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648297+0000", 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7780000000000001e-06 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648300+0000", 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8729999999999999e-06 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51234 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.649155+0000", 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.00360159, 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053204630000000001, 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51234 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649155+0000", 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649155+0000", 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649155+0000", 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0699999999999997e-07 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649159+0000", 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2649999999999998e-06 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649166+0000", 2024-04-23T23:50:25.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.815e-06 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649171+0000", 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.096e-06 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51043 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644634+0000", 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0081226579999999996, 2024-04-23T23:50:25.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00986918, 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51043 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644634+0000", 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644634+0000", 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644634+0000", 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644635+0000", 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0249999999999999e-06 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644638+0000", 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9380000000000001e-06 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644641+0000", 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3400000000000002e-06 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.858 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51075 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645329+0000", 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074269230000000002, 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092002809999999994, 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51075 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645329+0000", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645329+0000", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645330+0000", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6800000000000002e-07 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645331+0000", 2024-04-23T23:50:25.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4160000000000001e-06 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645334+0000", 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4750000000000002e-06 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645337+0000", 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7750000000000001e-06 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51107 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646014+0000", 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0067426580000000003, 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085425999999999992, 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51107 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646014+0000", 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646014+0000", 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646014+0000", 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3000000000000002e-07 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646015+0000", 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4580000000000001e-06 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646019+0000", 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5030000000000002e-06 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646022+0000", 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.756e-06 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51139 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646808+0000", 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.005948259, 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077747939999999998, 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51139 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.865 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646808+0000", 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646808+0000", 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646808+0000", 2024-04-23T23:50:25.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646811+0000", 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6599999999999999e-06 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646816+0000", 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6419999999999998e-06 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646819+0000", 2024-04-23T23:50:25.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3160000000000001e-06 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51171 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647559+0000", 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0051972370000000004, 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007050321, 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.868 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51171 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647559+0000", 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647559+0000", 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647559+0000", 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.22e-07 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647560+0000", 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647565+0000", 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7469999999999996e-06 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647568+0000", 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5090000000000002e-06 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51203 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648304+0000", 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.004451862, 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063441130000000002, 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51203 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648304+0000", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648304+0000", 2024-04-23T23:50:25.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648305+0000", 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.39e-07 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648306+0000", 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.61e-06 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648311+0000", 2024-04-23T23:50:25.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2610000000000004e-06 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648316+0000", 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9479999999999999e-06 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51235 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.649180+0000", 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0035758349999999999, 2024-04-23T23:50:25.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054955849999999999, 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51235 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649180+0000", 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.649180+0000", 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650829+0000", 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016482179999999999 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650833+0000", 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989999999999999e-06 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650844+0000", 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1629e-05 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650849+0000", 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51044 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.877 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644656+0000", 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0081003189999999999, 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010048478, 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51044 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644656+0000", 2024-04-23T23:50:25.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644656+0000", 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644656+0000", 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4099999999999999e-07 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644658+0000", 2024-04-23T23:50:25.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.449e-06 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644661+0000", 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4350000000000001e-06 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644664+0000", 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.785e-06 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51076 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645340+0000", 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074157889999999999, 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093921379999999995, 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51076 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645340+0000", 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645340+0000", 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645341+0000", 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645342+0000", 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8400000000000002e-07 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645345+0000", 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7850000000000002e-06 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645348+0000", 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9289999999999998e-06 2024-04-23T23:50:25.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51108 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646025+0000", 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0067311999999999997, 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087342730000000007, 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51108 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646025+0000", 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646025+0000", 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646025+0000", 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3999999999999997e-07 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646026+0000", 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.02e-06 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646030+0000", 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1049999999999999e-06 2024-04-23T23:50:25.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646032+0000", 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5440000000000001e-06 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51140 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646839+0000", 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0059172390000000004, 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079469810000000005, 2024-04-23T23:50:25.887 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51140 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646839+0000", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646839+0000", 2024-04-23T23:50:25.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646840+0000", 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0500000000000004e-07 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646842+0000", 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2910000000000002e-06 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646850+0000", 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7149999999999998e-06 2024-04-23T23:50:25.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646855+0000", 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8540000000000002e-06 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51172 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647572+0000", 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0051840649999999999, 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072405020000000002, 2024-04-23T23:50:25.890 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51172 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647572+0000", 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647572+0000", 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647573+0000", 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8100000000000003e-07 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647575+0000", 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8279999999999999e-06 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647582+0000", 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.939e-06 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647587+0000", 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51204 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648328+0000", 2024-04-23T23:50:25.893 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0044280079999999998, 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065178969999999999, 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51204 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648328+0000", 2024-04-23T23:50:25.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648328+0000", 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648329+0000", 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2500000000000001e-07 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648331+0000", 2024-04-23T23:50:25.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9319999999999999e-06 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648343+0000", 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2279000000000001e-05 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648348+0000", 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7330000000000003e-06 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51236 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.650858+0000", 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0018983509999999999, 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040149010000000004, 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51236 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650858+0000", 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650858+0000", 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650858+0000", 2024-04-23T23:50:25.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5999999999999999e-07 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650864+0000", 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7440000000000004e-06 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650870+0000", 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5589999999999996e-06 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650874+0000", 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4360000000000004e-06 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51045 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644696+0000", 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0080600749999999999, 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010204559, 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51045 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644696+0000", 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644696+0000", 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644697+0000", 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1699999999999999e-07 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644698+0000", 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.37e-06 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644701+0000", 2024-04-23T23:50:25.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4419999999999998e-06 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644704+0000", 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51077 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645351+0000", 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0074049240000000002, 2024-04-23T23:50:25.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0095760870000000005, 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51077 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645351+0000", 2024-04-23T23:50:25.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645351+0000", 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645352+0000", 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.89e-07 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645353+0000", 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.083e-06 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645356+0000", 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.959e-06 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645359+0000", 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6909999999999999e-06 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51109 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646042+0000", 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0067139610000000001, 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089115289999999996, 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51109 2024-04-23T23:50:25.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646042+0000", 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646042+0000", 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646043+0000", 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.41e-07 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646044+0000", 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.37e-06 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646047+0000", 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646050+0000", 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7310000000000002e-06 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51141 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646862+0000", 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0058945279999999996, 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081194849999999992, 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51141 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646862+0000", 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646862+0000", 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646862+0000", 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646867+0000", 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4209999999999997e-06 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646870+0000", 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9670000000000003e-06 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646873+0000", 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8090000000000001e-06 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51173 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647592+0000", 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0051638659999999996, 2024-04-23T23:50:25.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074153259999999999, 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51173 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647592+0000", 2024-04-23T23:50:25.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647592+0000", 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647593+0000", 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9400000000000001e-07 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647595+0000", 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7689999999999999e-06 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647600+0000", 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7010000000000001e-06 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647605+0000", 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.13e-06 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51205 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648369+0000", 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0043870599999999999, 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066648940000000002, 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51205 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648369+0000", 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648369+0000", 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648370+0000", 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.34e-07 2024-04-23T23:50:25.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648372+0000", 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4959999999999999e-06 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648376+0000", 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8179999999999997e-06 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648380+0000", 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7759999999999999e-06 2024-04-23T23:50:25.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51237 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.650894+0000", 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001862421, 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041667759999999996, 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51237 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650894+0000", 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650894+0000", 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650894+0000", 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.82e-07 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650897+0000", 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2570000000000002e-06 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650903+0000", 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0190000000000003e-06 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650908+0000", 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5249999999999996e-06 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51046 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644708+0000", 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0080484460000000008, 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01038188, 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51046 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644708+0000", 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644708+0000", 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644708+0000", 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.28e-07 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644711+0000", 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9869999999999999e-06 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644714+0000", 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.191e-06 2024-04-23T23:50:25.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644717+0000", 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1829999999999998e-06 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51078 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645363+0000", 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0073937109999999999, 2024-04-23T23:50:25.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097534709999999997, 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51078 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645363+0000", 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645363+0000", 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645363+0000", 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.28e-07 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645364+0000", 2024-04-23T23:50:25.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.052e-06 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645367+0000", 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4759999999999998e-06 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645371+0000", 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.242e-06 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51110 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646053+0000", 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0067030450000000004, 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090890520000000002, 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51110 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646053+0000", 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646053+0000", 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646053+0000", 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646054+0000", 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8400000000000002e-07 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646057+0000", 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.993e-06 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646060+0000", 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5770000000000001e-06 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.932 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51142 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646877+0000", 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0058792549999999999, 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082916919999999998, 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.933 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51142 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646877+0000", 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646877+0000", 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646877+0000", 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5699999999999999e-07 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646878+0000", 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7300000000000004e-07 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646882+0000", 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9090000000000002e-06 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646885+0000", 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7259999999999998e-06 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51174 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647627+0000", 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0051294180000000002, 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075683399999999998, 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51174 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647627+0000", 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647627+0000", 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647627+0000", 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6e-07 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647629+0000", 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5149999999999999e-06 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647632+0000", 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8290000000000001e-06 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647635+0000", 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.898e-06 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51206 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648384+0000", 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0043719350000000004, 2024-04-23T23:50:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068371170000000002, 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51206 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648384+0000", 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648384+0000", 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648385+0000", 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0100000000000005e-07 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648386+0000", 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6780000000000001e-06 2024-04-23T23:50:25.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648390+0000", 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6739999999999999e-06 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648394+0000", 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1690000000000002e-06 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51238 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.650915+0000", 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0018409450000000001, 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043325849999999999, 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51238 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650915+0000", 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650915+0000", 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650916+0000", 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8500000000000002e-07 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650919+0000", 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6600000000000001e-06 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650927+0000", 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0909999999999993e-06 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650932+0000", 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9869999999999999e-06 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51047 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644741+0000", 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0080148870000000001, 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010534363, 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51047 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644741+0000", 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644741+0000", 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644742+0000", 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3700000000000001e-07 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644743+0000", 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4130000000000001e-06 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644747+0000", 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.552e-06 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644749+0000", 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7360000000000001e-06 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51079 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645389+0000", 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0073668919999999999, 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099133989999999998, 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51079 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645389+0000", 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645389+0000", 2024-04-23T23:50:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645390+0000", 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645391+0000", 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3820000000000001e-06 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645395+0000", 2024-04-23T23:50:25.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5470000000000001e-06 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645397+0000", 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6529999999999998e-06 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51111 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646080+0000", 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0066766430000000003, 2024-04-23T23:50:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092495589999999992, 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51111 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646080+0000", 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646080+0000", 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646080+0000", 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646081+0000", 2024-04-23T23:50:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3680000000000001e-06 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646085+0000", 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.732e-06 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646088+0000", 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7070000000000001e-06 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51143 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646889+0000", 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0058675649999999999, 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084669299999999992, 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51143 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646889+0000", 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646889+0000", 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646889+0000", 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1399999999999998e-07 2024-04-23T23:50:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646891+0000", 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5340000000000001e-06 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646895+0000", 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646899+0000", 2024-04-23T23:50:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.269e-06 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51175 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647639+0000", 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0051176340000000002, 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077433270000000004, 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51175 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647639+0000", 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647639+0000", 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647639+0000", 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647640+0000", 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1820000000000001e-06 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647645+0000", 2024-04-23T23:50:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1270000000000002e-06 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647648+0000", 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51207 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648406+0000", 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0043506899999999999, 2024-04-23T23:50:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070028429999999999, 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51207 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648406+0000", 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648406+0000", 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648406+0000", 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9400000000000001e-07 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648408+0000", 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9549999999999999e-06 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648414+0000", 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648419+0000", 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4009999999999996e-06 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51239 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.650941+0000", 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001815472, 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044940479999999996, 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51239 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650941+0000", 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650941+0000", 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650941+0000", 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.08e-07 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650943+0000", 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9309999999999998e-06 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650953+0000", 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5580000000000006e-06 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650959+0000", 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7850000000000003e-06 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51048 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644781+0000", 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0079752910000000007, 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010681418, 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51048 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644781+0000", 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644781+0000", 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644781+0000", 2024-04-23T23:50:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-07 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644783+0000", 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.331e-06 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644786+0000", 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4410000000000002e-06 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644789+0000", 2024-04-23T23:50:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7580000000000001e-06 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51080 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645401+0000", 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0073548909999999997, 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0100873, 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51080 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645401+0000", 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645401+0000", 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645402+0000", 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645403+0000", 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9600000000000008e-07 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645406+0000", 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1269999999999999e-06 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645409+0000", 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.439e-06 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51112 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646091+0000", 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0066653069999999997, 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094242990000000006, 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51112 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646091+0000", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646091+0000", 2024-04-23T23:50:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646091+0000", 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0800000000000001e-07 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646092+0000", 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0079999999999999e-06 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646095+0000", 2024-04-23T23:50:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0019999999999998e-06 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646098+0000", 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7609999999999999e-06 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51144 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646913+0000", 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0058434170000000001, 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086287380000000004, 2024-04-23T23:50:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51144 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646913+0000", 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646913+0000", 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646913+0000", 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646915+0000", 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8759999999999999e-06 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646921+0000", 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0229999999999998e-06 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646926+0000", 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4510000000000002e-06 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51176 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647661+0000", 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0050947780000000003, 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079065809999999993, 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51176 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647661+0000", 2024-04-23T23:50:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647661+0000", 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647662+0000", 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1199999999999998e-07 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647664+0000", 2024-04-23T23:50:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.928e-06 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647670+0000", 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9529999999999996e-06 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647675+0000", 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3310000000000003e-06 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51208 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648443+0000", 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0043129409999999998, 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071512490000000001, 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51208 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648443+0000", 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648443+0000", 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648444+0000", 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6800000000000002e-07 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648445+0000", 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.652e-06 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648449+0000", 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9790000000000004e-06 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648452+0000", 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8990000000000001e-06 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51240 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.650967+0000", 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0017893519999999999, 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046538780000000002, 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51240 2024-04-23T23:50:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650967+0000", 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650967+0000", 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650967+0000", 2024-04-23T23:50:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.27e-07 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650969+0000", 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.074e-06 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650975+0000", 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.981e-06 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650980+0000", 2024-04-23T23:50:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4789999999999998e-06 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51049 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644792+0000", 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.007964179, 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01085591, 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51049 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644792+0000", 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644792+0000", 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644792+0000", 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0800000000000001e-07 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644793+0000", 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-07 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644796+0000", 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9399999999999998e-06 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644799+0000", 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.632e-06 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51081 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645418+0000", 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0073385610000000004, 2024-04-23T23:50:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010256802000000001, 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51081 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645418+0000", 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645418+0000", 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645418+0000", 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6600000000000002e-07 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645419+0000", 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4079999999999999e-06 2024-04-23T23:50:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645423+0000", 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5250000000000001e-06 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645426+0000", 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7949999999999999e-06 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51113 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646101+0000", 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0066554370000000002, 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096000970000000001, 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51113 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646101+0000", 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646101+0000", 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646101+0000", 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5600000000000001e-07 2024-04-23T23:50:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646102+0000", 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646105+0000", 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646109+0000", 2024-04-23T23:50:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.625e-06 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51145 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646946+0000", 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0058104580000000001, 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.008782293, 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51145 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646946+0000", 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646946+0000", 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646946+0000", 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646947+0000", 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.302e-06 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646951+0000", 2024-04-23T23:50:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8190000000000002e-06 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646956+0000", 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8230000000000002e-06 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51177 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647699+0000", 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0050575530000000002, 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080566189999999992, 2024-04-23T23:50:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51177 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647699+0000", 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647699+0000", 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647699+0000", 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647703+0000", 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8060000000000001e-06 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647707+0000", 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9419999999999997e-06 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647710+0000", 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6399999999999999e-06 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51209 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648456+0000", 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0043004929999999999, 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073260360000000002, 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51209 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648456+0000", 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648456+0000", 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648456+0000", 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.32e-07 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648458+0000", 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5579999999999999e-06 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648462+0000", 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4000000000000002e-06 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648465+0000", 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.819e-06 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51241 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.650988+0000", 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0017687460000000001, 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048205840000000002, 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51241 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650988+0000", 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650988+0000", 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650988+0000", 2024-04-23T23:50:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650990+0000", 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8470000000000001e-06 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.650997+0000", 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9439999999999999e-06 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651001+0000", 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6999999999999999e-06 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51050 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644838+0000", 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0079177880000000003, 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011002923, 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51050 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644838+0000", 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644838+0000", 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644839+0000", 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3600000000000004e-07 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644841+0000", 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.362e-06 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644847+0000", 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.609e-06 2024-04-23T23:50:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644852+0000", 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2340000000000002e-06 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51082 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645429+0000", 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0073272370000000003, 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010440043, 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51082 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645429+0000", 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645429+0000", 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645429+0000", 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.23e-07 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645432+0000", 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.123e-06 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645436+0000", 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1470000000000001e-06 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645438+0000", 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.729e-06 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51114 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646113+0000", 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0066437420000000002, 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009783234, 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51114 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646113+0000", 2024-04-23T23:50:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646113+0000", 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646113+0000", 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3099999999999999e-07 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646114+0000", 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.049e-06 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646117+0000", 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0340000000000001e-06 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646119+0000", 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4880000000000001e-06 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51146 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646974+0000", 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0057818670000000004, 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089482510000000008, 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51146 2024-04-23T23:50:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646974+0000", 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646974+0000", 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646975+0000", 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8599999999999998e-07 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646977+0000", 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.951e-06 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646983+0000", 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.032e-06 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646987+0000", 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3440000000000003e-06 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51178 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647734+0000", 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0050226849999999998, 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082152070000000004, 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51178 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647734+0000", 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647734+0000", 2024-04-23T23:50:26.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647734+0000", 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1600000000000001e-07 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647736+0000", 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9309999999999998e-06 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647742+0000", 2024-04-23T23:50:26.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1e-06 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647747+0000", 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4769999999999997e-06 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51210 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648479+0000", 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0042773209999999997, 2024-04-23T23:50:26.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074961539999999997, 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51210 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648479+0000", 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648479+0000", 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648479+0000", 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6700000000000003e-07 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648481+0000", 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9410000000000001e-06 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648488+0000", 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8650000000000003e-06 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648493+0000", 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2119999999999997e-06 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51242 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.033 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651012+0000", 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001743886, 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049889849999999996, 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51242 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651012+0000", 2024-04-23T23:50:26.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651012+0000", 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651013+0000", 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0499999999999999e-07 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651015+0000", 2024-04-23T23:50:26.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7880000000000001e-06 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651022+0000", 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3170000000000004e-06 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651028+0000", 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.772e-06 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51051 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644874+0000", 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0078826420000000005, 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011155019, 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51051 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644874+0000", 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644874+0000", 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644874+0000", 2024-04-23T23:50:26.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644876+0000", 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.419e-06 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644880+0000", 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6219999999999998e-06 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644883+0000", 2024-04-23T23:50:26.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.768e-06 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51083 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645465+0000", 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0072913279999999997, 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010590351, 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51083 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645465+0000", 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645465+0000", 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645465+0000", 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2199999999999999e-07 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645467+0000", 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645471+0000", 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9439999999999998e-06 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645473+0000", 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7599999999999998e-06 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51115 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646144+0000", 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0066121510000000001, 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099381580000000008, 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51115 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646144+0000", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646144+0000", 2024-04-23T23:50:26.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646145+0000", 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2300000000000002e-07 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646146+0000", 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.466e-06 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646150+0000", 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6049999999999998e-06 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646152+0000", 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7130000000000001e-06 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51147 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647011+0000", 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0057450299999999999, 2024-04-23T23:50:26.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090979849999999994, 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51147 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647011+0000", 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647011+0000", 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647011+0000", 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5600000000000002e-07 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647013+0000", 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.621e-06 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647017+0000", 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.799e-06 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647020+0000", 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9229999999999998e-06 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51179 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.049 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647770+0000", 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0049859479999999996, 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083652359999999999, 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51179 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647770+0000", 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647770+0000", 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647771+0000", 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9299999999999999e-07 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647772+0000", 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6339999999999999e-06 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647776+0000", 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9809999999999997e-06 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647779+0000", 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8940000000000002e-06 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51211 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648514+0000", 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0042424259999999997, 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076480899999999997, 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51211 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648514+0000", 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648514+0000", 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648514+0000", 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5899999999999998e-07 2024-04-23T23:50:26.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648516+0000", 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.612e-06 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648520+0000", 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8920000000000002e-06 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648523+0000", 2024-04-23T23:50:26.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9210000000000001e-06 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51243 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651034+0000", 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0017223609999999999, 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051542979999999999, 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51243 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651034+0000", 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651034+0000", 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651034+0000", 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000001e-07 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651036+0000", 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.95e-06 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651042+0000", 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8499999999999999e-06 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651047+0000", 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6260000000000003e-06 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51052 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644886+0000", 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0078699579999999998, 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011328917000000001, 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51052 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644886+0000", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644886+0000", 2024-04-23T23:50:26.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644887+0000", 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7899999999999999e-07 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644888+0000", 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.035e-06 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644891+0000", 2024-04-23T23:50:26.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9799999999999998e-06 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644893+0000", 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8109999999999999e-06 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51084 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645477+0000", 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0072793689999999999, 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010764708, 2024-04-23T23:50:26.062 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51084 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645477+0000", 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645477+0000", 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645477+0000", 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1500000000000001e-07 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645479+0000", 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9099999999999999e-06 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645482+0000", 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1089999999999998e-06 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645485+0000", 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6460000000000002e-06 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51116 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.065 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646336+0000", 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0064198329999999998, 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099314590000000001, 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51116 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646336+0000", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646336+0000", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646337+0000", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1500000000000005e-07 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646340+0000", 2024-04-23T23:50:26.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0019999999999998e-06 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646345+0000", 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0799999999999996e-06 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646349+0000", 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.827e-06 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51148 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647024+0000", 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0057321230000000004, 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.009270136, 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51148 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647024+0000", 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647024+0000", 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647024+0000", 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1299999999999999e-07 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647026+0000", 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5149999999999999e-06 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647031+0000", 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8629999999999999e-06 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647034+0000", 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2179999999999999e-06 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.071 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51180 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647783+0000", 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0049731799999999998, 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085382630000000008, 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51180 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647783+0000", 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647783+0000", 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647783+0000", 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.41e-07 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647785+0000", 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1969999999999999e-06 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647790+0000", 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1569999999999999e-06 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647792+0000", 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6390000000000001e-06 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51212 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648527+0000", 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0042294089999999999, 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078211429999999992, 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.075 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51212 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648527+0000", 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648527+0000", 2024-04-23T23:50:26.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648527+0000", 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648529+0000", 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6109999999999999e-06 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648533+0000", 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3420000000000001e-06 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648538+0000", 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4789999999999998e-06 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51244 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.078 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651052+0000", 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001704094, 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053221450000000003, 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51244 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651052+0000", 2024-04-23T23:50:26.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651052+0000", 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651053+0000", 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7899999999999999e-07 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651054+0000", 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.719e-06 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651060+0000", 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7200000000000003e-06 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651069+0000", 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6419999999999996e-06 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51053 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644896+0000", 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0078597719999999992, 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011504923, 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51053 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.082 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644896+0000", 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644896+0000", 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644897+0000", 2024-04-23T23:50:26.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1899999999999999e-07 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644898+0000", 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8299999999999995e-07 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644901+0000", 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0350000000000002e-06 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644903+0000", 2024-04-23T23:50:26.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.604e-06 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51085 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645511+0000", 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0072449869999999996, 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010916555999999999, 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51085 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645511+0000", 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645511+0000", 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645512+0000", 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.34e-07 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645513+0000", 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.395e-06 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645517+0000", 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.455e-06 2024-04-23T23:50:26.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645519+0000", 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7690000000000001e-06 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51117 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646356+0000", 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0063999779999999997, 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010099221, 2024-04-23T23:50:26.088 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51117 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646356+0000", 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646356+0000", 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646357+0000", 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4900000000000002e-07 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646363+0000", 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6429999999999998e-06 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646369+0000", 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9880000000000001e-06 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646373+0000", 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6449999999999998e-06 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.091 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51149 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647048+0000", 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0057085690000000001, 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094587209999999998, 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51149 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647048+0000", 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647048+0000", 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647048+0000", 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647050+0000", 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9039999999999999e-06 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647058+0000", 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6109999999999996e-06 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647065+0000", 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2729999999999997e-06 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51181 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647797+0000", 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0049591610000000001, 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087363370000000003, 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51181 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.095 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647797+0000", 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647797+0000", 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647798+0000", 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9299999999999998e-07 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647799+0000", 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.778e-06 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647805+0000", 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.682e-06 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647809+0000", 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4560000000000002e-06 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51213 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648550+0000", 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0042058479999999999, 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080093639999999997, 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51213 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648550+0000", 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648550+0000", 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648551+0000", 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3799999999999998e-07 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648556+0000", 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9459999999999997e-06 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648563+0000", 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648567+0000", 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4680000000000003e-06 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51245 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651075+0000", 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0016815529999999999, 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055123890000000004, 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51245 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651075+0000", 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651075+0000", 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651075+0000", 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651077+0000", 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9379999999999999e-06 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651083+0000", 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8479999999999997e-06 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651088+0000", 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.135e-06 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.104 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51054 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644906+0000", 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0078499099999999999, 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011708087000000001, 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.105 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51054 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644906+0000", 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644906+0000", 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644907+0000", 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644908+0000", 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.111e-06 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644911+0000", 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.994e-06 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644913+0000", 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51086 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645523+0000", 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0072337369999999996, 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01111936, 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51086 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645523+0000", 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645523+0000", 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645523+0000", 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645524+0000", 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5999999999999991e-07 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645527+0000", 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9210000000000001e-06 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645530+0000", 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2820000000000001e-06 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51118 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.111 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646388+0000", 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0063682249999999999, 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010280789, 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51118 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646388+0000", 2024-04-23T23:50:26.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646388+0000", 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646389+0000", 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8999999999999996e-07 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646391+0000", 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.165e-06 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646397+0000", 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2650000000000002e-06 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646402+0000", 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4780000000000002e-06 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51150 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647087+0000", 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0056696430000000003, 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096093749999999999, 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51150 2024-04-23T23:50:26.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647087+0000", 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647087+0000", 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647087+0000", 2024-04-23T23:50:26.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8099999999999999e-07 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647089+0000", 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.588e-06 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647093+0000", 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8500000000000004e-06 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647097+0000", 2024-04-23T23:50:26.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.332e-06 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51182 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647815+0000", 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0049412579999999996, 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089071919999999995, 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51182 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647815+0000", 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647815+0000", 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647815+0000", 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-07 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647817+0000", 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6950000000000001e-06 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647830+0000", 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3210999999999999e-05 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647835+0000", 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8760000000000004e-06 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51214 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648591+0000", 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0041647489999999997, 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081574260000000006, 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51214 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648591+0000", 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648591+0000", 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648592+0000", 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8500000000000002e-07 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648593+0000", 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.657e-06 2024-04-23T23:50:26.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648597+0000", 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9489999999999998e-06 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648600+0000", 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8329999999999998e-06 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51246 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651094+0000", 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0016624980000000001, 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056813419999999998, 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51246 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651094+0000", 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651094+0000", 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651094+0000", 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9400000000000001e-07 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651096+0000", 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8190000000000001e-06 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651103+0000", 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2570000000000001e-06 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651109+0000", 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3809999999999998e-06 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.127 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51055 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644937+0000", 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0078188460000000008, 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011864949, 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51055 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644937+0000", 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644937+0000", 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644938+0000", 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6300000000000001e-07 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644939+0000", 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4440000000000001e-06 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644943+0000", 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8399999999999997e-06 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644946+0000", 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7329999999999999e-06 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51087 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645557+0000", 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0071990379999999996, 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011282825999999999, 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51087 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645557+0000", 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645557+0000", 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645558+0000", 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.32e-07 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645559+0000", 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4100000000000001e-06 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645563+0000", 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5049999999999999e-06 2024-04-23T23:50:26.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645565+0000", 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7860000000000001e-06 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51119 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646427+0000", 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0063288859999999997, 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010440288000000001, 2024-04-23T23:50:26.134 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51119 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646427+0000", 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646427+0000", 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646428+0000", 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1100000000000002e-07 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646429+0000", 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.809e-06 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646434+0000", 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.036e-06 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646437+0000", 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0450000000000001e-06 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51151 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.137 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647101+0000", 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0056552829999999997, 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097930629999999994, 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51151 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647101+0000", 2024-04-23T23:50:26.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647101+0000", 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647101+0000", 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0900000000000002e-07 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647103+0000", 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647107+0000", 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2529999999999998e-06 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647111+0000", 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.411e-06 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51183 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647841+0000", 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0049147870000000003, 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090792730000000006, 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51183 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647841+0000", 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647841+0000", 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647842+0000", 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6899999999999998e-07 2024-04-23T23:50:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647845+0000", 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7250000000000002e-06 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647848+0000", 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9360000000000001e-06 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647851+0000", 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6259999999999999e-06 2024-04-23T23:50:26.143 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51215 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648604+0000", 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0041524589999999998, 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083431379999999999, 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.144 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51215 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648604+0000", 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648604+0000", 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648604+0000", 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.28e-07 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648605+0000", 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.252e-06 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648611+0000", 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5600000000000001e-06 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648614+0000", 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7080000000000002e-06 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51247 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651125+0000", 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001631333, 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058482689999999997, 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.147 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51247 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651125+0000", 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651125+0000", 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651125+0000", 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2500000000000001e-07 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651127+0000", 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.513e-06 2024-04-23T23:50:26.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651132+0000", 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9559999999999996e-06 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651135+0000", 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0510000000000001e-06 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51056 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644950+0000", 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0078065850000000004, 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012051009999999999, 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51056 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644950+0000", 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644950+0000", 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644950+0000", 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2600000000000001e-07 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644951+0000", 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.283e-06 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644954+0000", 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1200000000000002e-06 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644957+0000", 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6850000000000001e-06 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51088 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645569+0000", 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0071875569999999998, 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011458492000000001, 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51088 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645569+0000", 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645569+0000", 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645569+0000", 2024-04-23T23:50:26.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645570+0000", 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0869999999999999e-06 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645573+0000", 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9629999999999998e-06 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645576+0000", 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1779999999999999e-06 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51120 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646441+0000", 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0063147560000000004, 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010612224, 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.157 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51120 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646441+0000", 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646441+0000", 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646442+0000", 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5499999999999999e-07 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646443+0000", 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-06 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646448+0000", 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646451+0000", 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0819999999999999e-06 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51152 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647123+0000", 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.005633122, 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099568899999999995, 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51152 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647123+0000", 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647123+0000", 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647124+0000", 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.15e-07 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647126+0000", 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1339999999999998e-06 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647131+0000", 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6960000000000002e-06 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647136+0000", 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0320000000000003e-06 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51184 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647855+0000", 2024-04-23T23:50:26.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0049015400000000002, 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092517369999999995, 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51184 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647855+0000", 2024-04-23T23:50:26.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647855+0000", 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647855+0000", 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4999999999999999e-07 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647856+0000", 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1769999999999999e-06 2024-04-23T23:50:26.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647859+0000", 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.179e-06 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647862+0000", 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6539999999999999e-06 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51216 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648627+0000", 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.004129097, 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085058080000000001, 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51216 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648627+0000", 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648627+0000", 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648628+0000", 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000001e-07 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648631+0000", 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1750000000000001e-06 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648637+0000", 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6320000000000002e-06 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648642+0000", 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3089999999999997e-06 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51248 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651138+0000", 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001617788, 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060206799999999996, 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51248 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651138+0000", 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651138+0000", 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651139+0000", 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4200000000000002e-07 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651143+0000", 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651147+0000", 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4979999999999998e-06 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651151+0000", 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0840000000000002e-06 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51057 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644961+0000", 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0077951360000000003, 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012225290999999999, 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51057 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644961+0000", 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644961+0000", 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644961+0000", 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6199999999999999e-07 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644962+0000", 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0219999999999999e-06 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644966+0000", 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0280000000000001e-06 2024-04-23T23:50:26.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644968+0000", 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6410000000000002e-06 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51089 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645602+0000", 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0071541069999999998, 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011611151, 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51089 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645602+0000", 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645602+0000", 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645603+0000", 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4799999999999999e-07 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645604+0000", 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3519999999999999e-06 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645607+0000", 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3809999999999999e-06 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645610+0000", 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1540000000000002e-06 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51121 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646455+0000", 2024-04-23T23:50:26.179 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0063009110000000002, 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010784436, 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51121 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646455+0000", 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646455+0000", 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646456+0000", 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646458+0000", 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7349999999999999e-06 2024-04-23T23:50:26.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646463+0000", 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8320000000000002e-06 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646469+0000", 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5509999999999999e-06 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.182 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51153 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647161+0000", 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0055947840000000002, 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010104821999999999, 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51153 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647161+0000", 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647161+0000", 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647162+0000", 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5199999999999998e-07 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647170+0000", 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9899999999999997e-06 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647179+0000", 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1570000000000006e-06 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647183+0000", 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8689999999999997e-06 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51185 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647865+0000", 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0048914500000000003, 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094279529999999993, 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51185 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647865+0000", 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647865+0000", 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647865+0000", 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:26.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647866+0000", 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9800000000000002e-07 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647870+0000", 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1049999999999997e-06 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647875+0000", 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6890000000000003e-06 2024-04-23T23:50:26.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51217 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648664+0000", 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0040920899999999996, 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086556740000000004, 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.189 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51217 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648664+0000", 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648664+0000", 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648664+0000", 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0800000000000001e-07 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648666+0000", 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.764e-06 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648670+0000", 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9790000000000004e-06 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648673+0000", 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.802e-06 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51249 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651154+0000", 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001602104, 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061918040000000004, 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51249 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651154+0000", 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651154+0000", 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651154+0000", 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5899999999999998e-07 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651156+0000", 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.575e-06 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651159+0000", 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2049999999999998e-06 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651162+0000", 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7159999999999999e-06 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51058 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644971+0000", 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.007785073, 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012401870000000001, 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.195 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51058 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644971+0000", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644971+0000", 2024-04-23T23:50:26.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644971+0000", 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644979+0000", 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8520000000000004e-06 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644983+0000", 2024-04-23T23:50:26.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6339999999999999e-06 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644986+0000", 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2119999999999999e-06 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51090 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645614+0000", 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0071421089999999998, 2024-04-23T23:50:26.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011785385000000001, 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51090 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645614+0000", 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645614+0000", 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645614+0000", 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.23e-07 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645615+0000", 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9600000000000008e-07 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645618+0000", 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.055e-06 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645621+0000", 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.201 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51122 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646474+0000", 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0062817760000000002, 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01095171, 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51122 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646474+0000", 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646474+0000", 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646475+0000", 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9900000000000001e-07 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646477+0000", 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.739e-06 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646482+0000", 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.631e-06 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646489+0000", 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.968e-06 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51154 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647188+0000", 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0055678749999999999, 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010264082000000001, 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51154 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.205 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647188+0000", 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647188+0000", 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647189+0000", 2024-04-23T23:50:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9100000000000004e-07 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647191+0000", 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9300000000000002e-06 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647197+0000", 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8170000000000004e-06 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647201+0000", 2024-04-23T23:50:26.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4390000000000002e-06 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51186 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647879+0000", 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0048775329999999999, 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096009170000000005, 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51186 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647879+0000", 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647879+0000", 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647879+0000", 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9399999999999995e-07 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647881+0000", 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.751e-06 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647887+0000", 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8470000000000001e-06 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647891+0000", 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3279999999999999e-06 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51218 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648678+0000", 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0040786329999999999, 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088282059999999999, 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51218 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648678+0000", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648678+0000", 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648678+0000", 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1600000000000001e-07 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648680+0000", 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7269999999999999e-06 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648683+0000", 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-04-23T23:50:26.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648686+0000", 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0589999999999998e-06 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51250 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651165+0000", 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001591458, 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063672210000000002, 2024-04-23T23:50:26.214 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51250 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651165+0000", 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651165+0000", 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651165+0000", 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9799999999999999e-07 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651166+0000", 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.034e-06 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651169+0000", 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.021e-06 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651172+0000", 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7800000000000001e-06 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51059 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.217 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.644990+0000", 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0077665130000000001, 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01256937, 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51059 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644990+0000", 2024-04-23T23:50:26.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644990+0000", 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644990+0000", 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644991+0000", 2024-04-23T23:50:26.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0279999999999999e-06 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644994+0000", 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1879999999999998e-06 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.644998+0000", 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0430000000000001e-06 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51091 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645640+0000", 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0071165919999999997, 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011946122999999999, 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51091 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645640+0000", 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645640+0000", 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645640+0000", 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645641+0000", 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.311e-06 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645645+0000", 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3900000000000002e-06 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645647+0000", 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6790000000000001e-06 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51123 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646495+0000", 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0062608580000000002, 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011116812, 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51123 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646495+0000", 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646495+0000", 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646496+0000", 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.96e-07 2024-04-23T23:50:26.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646497+0000", 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3009999999999999e-06 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646501+0000", 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7100000000000001e-06 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646503+0000", 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7719999999999999e-06 2024-04-23T23:50:26.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51155 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647211+0000", 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0055453350000000002, 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010427529, 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51155 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647211+0000", 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647211+0000", 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647211+0000", 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.01e-07 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647213+0000", 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9350000000000001e-06 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647219+0000", 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-06 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647223+0000", 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5170000000000001e-06 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51187 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647896+0000", 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0048600240000000001, 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097688159999999996, 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51187 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647896+0000", 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647896+0000", 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647897+0000", 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6100000000000002e-07 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647898+0000", 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6589999999999999e-06 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647905+0000", 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4330000000000001e-06 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647909+0000", 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8630000000000001e-06 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51219 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.233 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648699+0000", 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0040570830000000004, 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089925839999999996, 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51219 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648699+0000", 2024-04-23T23:50:26.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648699+0000", 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648700+0000", 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9299999999999999e-07 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648701+0000", 2024-04-23T23:50:26.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.88e-06 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648708+0000", 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5100000000000004e-06 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648712+0000", 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51251 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651175+0000", 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0015815340000000001, 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065433549999999998, 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51251 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651175+0000", 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651175+0000", 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651175+0000", 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1899999999999999e-07 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651176+0000", 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.054e-06 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651179+0000", 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.965e-06 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651182+0000", 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5689999999999999e-06 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51060 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645010+0000", 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0077462130000000001, 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012735081000000001, 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51060 2024-04-23T23:50:26.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645010+0000", 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645010+0000", 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645010+0000", 2024-04-23T23:50:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645012+0000", 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.933e-06 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645019+0000", 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5570000000000001e-06 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645023+0000", 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4610000000000001e-06 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51092 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645650+0000", 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0071058069999999996, 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012121274, 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51092 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645650+0000", 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645650+0000", 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645651+0000", 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0599999999999999e-07 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645652+0000", 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7399999999999991e-07 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645655+0000", 2024-04-23T23:50:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0690000000000002e-06 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645657+0000", 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51124 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646507+0000", 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0062495709999999998, 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011291697, 2024-04-23T23:50:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51124 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646507+0000", 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646507+0000", 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646507+0000", 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.05e-07 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646508+0000", 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0100000000000001e-06 2024-04-23T23:50:26.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646511+0000", 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0460000000000001e-06 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646514+0000", 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6220000000000001e-06 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51156 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647241+0000", 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.005515364, 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010588030999999999, 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51156 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647241+0000", 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647241+0000", 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647241+0000", 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0100000000000001e-07 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647243+0000", 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5960000000000001e-06 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647247+0000", 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1439999999999996e-06 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647250+0000", 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9780000000000001e-06 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51188 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647932+0000", 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0048246749999999996, 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099242500000000008, 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51188 2024-04-23T23:50:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647932+0000", 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647932+0000", 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647932+0000", 2024-04-23T23:50:26.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6e-07 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647934+0000", 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.705e-06 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647938+0000", 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.746e-06 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647941+0000", 2024-04-23T23:50:26.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8430000000000001e-06 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51220 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648736+0000", 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0040203490000000003, 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091461159999999993, 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51220 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648736+0000", 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648736+0000", 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648736+0000", 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5600000000000001e-07 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648738+0000", 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.581e-06 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648742+0000", 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8820000000000003e-06 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648744+0000", 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7089999999999998e-06 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51252 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651184+0000", 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0015718660000000001, 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067238749999999998, 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51252 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651184+0000", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651184+0000", 2024-04-23T23:50:26.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651185+0000", 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3799999999999999e-07 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651186+0000", 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0020000000000001e-06 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651189+0000", 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9579999999999999e-06 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651191+0000", 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5299999999999999e-06 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51061 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645038+0000", 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0077186169999999997, 2024-04-23T23:50:26.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012897860000000001, 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51061 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645038+0000", 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645038+0000", 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645038+0000", 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2e-07 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645040+0000", 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645043+0000", 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.755e-06 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645046+0000", 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.706e-06 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51093 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645676+0000", 2024-04-23T23:50:26.265 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.007080706, 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012286174, 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51093 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645676+0000", 2024-04-23T23:50:26.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645676+0000", 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645676+0000", 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8599999999999999e-07 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645677+0000", 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.328e-06 2024-04-23T23:50:26.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645681+0000", 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3900000000000002e-06 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645683+0000", 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7209999999999999e-06 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.268 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51125 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646516+0000", 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0062397599999999996, 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011471373, 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51125 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646516+0000", 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646516+0000", 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646517+0000", 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1199999999999999e-07 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646518+0000", 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.049e-06 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646521+0000", 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6890000000000002e-06 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646526+0000", 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989999999999999e-06 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51157 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647254+0000", 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0055025949999999999, 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010760498, 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51157 2024-04-23T23:50:26.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647254+0000", 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647254+0000", 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647254+0000", 2024-04-23T23:50:26.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647255+0000", 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2529999999999999e-06 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647260+0000", 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1070000000000004e-06 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647263+0000", 2024-04-23T23:50:26.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51189 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647965+0000", 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0047917339999999998, 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010075986, 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51189 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647965+0000", 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647965+0000", 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647965+0000", 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0500000000000004e-07 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647970+0000", 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2660000000000001e-06 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647976+0000", 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0480000000000004e-06 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647982+0000", 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6099999999999997e-06 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51221 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648749+0000", 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.004007381, 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093179950000000008, 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51221 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648749+0000", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648749+0000", 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648749+0000", 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5899999999999998e-07 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648750+0000", 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.353e-06 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648756+0000", 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:50:26.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648760+0000", 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0509999999999998e-06 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51253 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651194+0000", 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0015623169999999999, 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068991490000000003, 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51253 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651194+0000", 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651194+0000", 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651194+0000", 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.16e-07 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651195+0000", 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.569999999999999e-07 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651198+0000", 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.903e-06 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651201+0000", 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.627e-06 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51062 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645049+0000", 2024-04-23T23:50:26.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.007706872, 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013070913999999999, 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51062 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645049+0000", 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645049+0000", 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645050+0000", 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3999999999999998e-07 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645052+0000", 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080000000000002e-06 2024-04-23T23:50:26.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645055+0000", 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2260000000000001e-06 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645057+0000", 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.712e-06 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.287 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51094 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645686+0000", 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0070697620000000003, 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012460703, 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51094 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645686+0000", 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645686+0000", 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645687+0000", 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3900000000000001e-07 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645692+0000", 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2549999999999997e-06 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645695+0000", 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.41e-06 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645698+0000", 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7310000000000002e-06 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51126 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646531+0000", 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0062255929999999998, 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01164312, 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51126 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646531+0000", 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646531+0000", 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646531+0000", 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.13e-07 2024-04-23T23:50:26.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646533+0000", 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.618e-06 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646538+0000", 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6350000000000003e-06 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646543+0000", 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2740000000000001e-06 2024-04-23T23:50:26.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51158 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647276+0000", 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0054797550000000002, 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010924198, 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51158 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647276+0000", 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647276+0000", 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647277+0000", 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647279+0000", 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8780000000000001e-06 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647284+0000", 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5740000000000003e-06 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647291+0000", 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1210000000000003e-06 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51190 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648007+0000", 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0047494820000000002, 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010220434, 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.297 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51190 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648007+0000", 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648007+0000", 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648007+0000", 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7000000000000001e-07 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648009+0000", 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7260000000000001e-06 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648013+0000", 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0640000000000004e-06 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648016+0000", 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51222 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648773+0000", 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0039836089999999999, 2024-04-23T23:50:26.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094807680000000005, 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51222 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648773+0000", 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648773+0000", 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648773+0000", 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2600000000000002e-07 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648775+0000", 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9609999999999999e-06 2024-04-23T23:50:26.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648782+0000", 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5679999999999996e-06 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648786+0000", 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3590000000000001e-06 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51254 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651203+0000", 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.001552798, 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070762239999999999, 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51254 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651203+0000", 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651203+0000", 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651204+0000", 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651205+0000", 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.004e-06 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651208+0000", 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9950000000000001e-06 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651210+0000", 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5399999999999998e-06 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51063 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645086+0000", 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0076707030000000001, 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0132213, 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51063 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645086+0000", 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645086+0000", 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645086+0000", 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2e-07 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645088+0000", 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.787e-06 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645094+0000", 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0990000000000004e-06 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645097+0000", 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.129e-06 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51095 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645713+0000", 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.007043232, 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012621327, 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51095 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645713+0000", 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645713+0000", 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645713+0000", 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5400000000000002e-07 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645716+0000", 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4839999999999998e-06 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645719+0000", 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5200000000000002e-06 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645723+0000", 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6839999999999998e-06 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51127 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646548+0000", 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0062081519999999998, 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011812445, 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.313 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51127 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646548+0000", 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646548+0000", 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646548+0000", 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4499999999999998e-07 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646550+0000", 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.544e-06 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646556+0000", 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6829999999999996e-06 2024-04-23T23:50:26.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646560+0000", 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.997e-06 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51159 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647297+0000", 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0054596779999999999, 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011090173, 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51159 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647297+0000", 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647297+0000", 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647297+0000", 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-07 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647299+0000", 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.99e-06 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647305+0000", 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9939999999999997e-06 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647309+0000", 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0840000000000002e-06 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51191 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648019+0000", 2024-04-23T23:50:26.319 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0047368519999999997, 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01039443, 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51191 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648019+0000", 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648019+0000", 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648021+0000", 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648022+0000", 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6390000000000001e-06 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648027+0000", 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6349999999999997e-06 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648030+0000", 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5030000000000002e-06 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.322 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51223 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648811+0000", 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0039454989999999999, 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0096302560000000002, 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51223 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648811+0000", 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648811+0000", 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648811+0000", 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7000000000000001e-07 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648813+0000", 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.657e-06 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648817+0000", 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9580000000000001e-06 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648819+0000", 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51255 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651213+0000", 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0015432519999999999, 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072548730000000002, 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51255 2024-04-23T23:50:26.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651213+0000", 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651213+0000", 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651213+0000", 2024-04-23T23:50:26.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1500000000000001e-07 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651214+0000", 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0109999999999999e-06 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651217+0000", 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9689999999999999e-06 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651220+0000", 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3800000000000001e-06 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51064 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645132+0000", 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0076239680000000001, 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013362804000000001, 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51064 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645132+0000", 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645132+0000", 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645133+0000", 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645134+0000", 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.655e-06 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645138+0000", 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6320000000000001e-06 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645141+0000", 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9210000000000001e-06 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51096 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645729+0000", 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0070269089999999996, 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012792344000000001, 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51096 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645729+0000", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645729+0000", 2024-04-23T23:50:26.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645730+0000", 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645732+0000", 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9850000000000001e-06 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645735+0000", 2024-04-23T23:50:26.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4630000000000001e-06 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645738+0000", 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6819999999999999e-06 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51128 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646583+0000", 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0061732890000000002, 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011965849000000001, 2024-04-23T23:50:26.335 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51128 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646583+0000", 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646583+0000", 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646583+0000", 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6199999999999999e-07 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646586+0000", 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6579999999999998e-06 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646590+0000", 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.721e-06 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646592+0000", 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.88e-06 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.338 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51160 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647321+0000", 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0054348290000000004, 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011253992000000001, 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51160 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647321+0000", 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647321+0000", 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647322+0000", 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4400000000000001e-07 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647323+0000", 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5939999999999999e-06 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647327+0000", 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7950000000000001e-06 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647330+0000", 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8100000000000002e-06 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51192 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648043+0000", 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0047131430000000004, 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010558867, 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51192 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648043+0000", 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648043+0000", 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648044+0000", 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6000000000000004e-07 2024-04-23T23:50:26.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648046+0000", 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.95e-06 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648053+0000", 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0249999999999997e-06 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648057+0000", 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0550000000000001e-06 2024-04-23T23:50:26.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51224 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648841+0000", 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0039147649999999997, 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097872730000000008, 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.345 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51224 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648841+0000", 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648841+0000", 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648842+0000", 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9500000000000003e-07 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648844+0000", 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.063e-06 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648850+0000", 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2920000000000001e-06 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648855+0000", 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7380000000000002e-06 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51256 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651222+0000", 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0015339419999999999, 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074327439999999998, 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.348 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51256 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651222+0000", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651222+0000", 2024-04-23T23:50:26.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651223+0000", 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.03e-07 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651223+0000", 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9800000000000002e-07 2024-04-23T23:50:26.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651227+0000", 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4740000000000001e-06 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651229+0000", 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5349999999999999e-06 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51065 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645144+0000", 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0076121039999999997, 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013538424, 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51065 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645144+0000", 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645144+0000", 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645144+0000", 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.23e-07 2024-04-23T23:50:26.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645145+0000", 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0440000000000001e-06 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645149+0000", 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9480000000000001e-06 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645152+0000", 2024-04-23T23:50:26.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7580000000000001e-06 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51097 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.645750+0000", 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0070061430000000003, 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012958740999999999, 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51097 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645750+0000", 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645750+0000", 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645750+0000", 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6899999999999999e-07 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645752+0000", 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.494e-06 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645755+0000", 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5580000000000001e-06 2024-04-23T23:50:26.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.645758+0000", 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8490000000000002e-06 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51129 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.646597+0000", 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0061595110000000003, 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012138278000000001, 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51129 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646597+0000", 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646597+0000", 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646597+0000", 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2300000000000001e-07 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646603+0000", 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4399999999999996e-06 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646608+0000", 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.646612+0000", 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6949999999999999e-06 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51161 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.647333+0000", 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0054231720000000004, 2024-04-23T23:50:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011428305, 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51161 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647333+0000", 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647333+0000", 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647333+0000", 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0699999999999998e-07 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647336+0000", 2024-04-23T23:50:26.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3860000000000001e-06 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647341+0000", 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9640000000000002e-06 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.647344+0000", 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51193 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648081+0000", 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0046751140000000002, 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010706919000000001, 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51193 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648081+0000", 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648081+0000", 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648081+0000", 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5199999999999998e-07 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648083+0000", 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.638e-06 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648087+0000", 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2270000000000004e-06 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648090+0000", 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.367 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51225 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.648868+0000", 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0038880640000000001, 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099463999999999993, 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51225 2024-04-23T23:50:26.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648868+0000", 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648868+0000", 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648869+0000", 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-07 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648871+0000", 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.125e-06 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648879+0000", 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3489999999999992e-06 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.648884+0000", 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5990000000000004e-06 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:51257 3.5 3.e78bdc05 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.651232+0000", 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.0015239889999999999, 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076171800000000003, 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:26.371 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 51257 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651232+0000", 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.651232+0000", 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.652835+0000", 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001603204 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.652839+0000", 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.652845+0000", 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1759999999999998e-06 2024-04-23T23:50:26.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.652852+0000", 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1399999999999997e-06 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 224 2024-04-23T23:50:26.374 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:26.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:26.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2897 read 0x548448 thru 0x558295 (0xfe4e bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2898 write 0xbaa4384 thru 0xbab2b20 (0xe79d bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2899 read 0x6b075b2 thru 0x6b09b6b (0x25ba bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2900 write 0x37c3e66 thru 0x37cab0c (0x6ca7 bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2902 read 0xb726888 thru 0xb72c383 (0x5afc bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2903 write 0x9f82f8a thru 0x9f8f9d3 (0xca4a bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2906 punch from 0x79836ae to 0x79875a0, (0x3ef2 bytes) 2024-04-23T23:50:26.380 INFO:teuthology.orchestra.run.smithi092.stdout:2907 punch from 0x3ea3250 to 0x3ead2fa, (0xa0aa bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2908 trunc from 0xbab2b21 to 0xdfe7f6 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2909 trunc from 0xdfe7f6 to 0xdbc79e7 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2911 write 0x7486029 thru 0x74954d4 (0xf4ac bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2912 read 0x6710266 thru 0x6710fb0 (0xd4b bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2915 write 0x8b147a8 thru 0x8b1f667 (0xaec0 bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2916 trunc from 0xdbc79e7 to 0x27dad9f 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2917 writesame 0x28e0151 thru 0x28e0c01 data_size 0x1(0xab1 bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2918 punch from 0x26e101 to 0x27761b, (0x951a bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2919 writesame 0xceeb15e thru 0xcef5ad2 data_size 0xd(0xa975 bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2920 trunc from 0xcef5ad3 to 0x6d7e9a2 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2921 writesame 0x8578bcc thru 0x857f649 data_size 0x2b(0x6a7e bytes) 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.381 INFO:teuthology.orchestra.run.smithi092.stdout:2923 read 0x806ea6c thru 0x80775d3 (0x8b68 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2925 write 0x415d2e3 thru 0x415dfd4 (0xcf2 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2926 punch from 0x3a8f3f2 to 0x3a94ca6, (0x58b4 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2931 trunc from 0x857f64a to 0x5e7cbbc 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2932 write 0xa00a95a thru 0xa00b262 (0x909 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2934 trunc from 0xa00b263 to 0x701f783 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2935 trunc from 0x701f783 to 0x3b850ed 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2938 trunc from 0x3b850ed to 0x3b218fe 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2939 read 0x2f51f01 thru 0x2f58d27 (0x6e27 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2943 read 0xe58ce thru 0xebe89 (0x65bc bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2944 write 0x492446b thru 0x4930a4f (0xc5e5 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2945 write 0x818f0a0 thru 0x8196e6e (0x7dcf bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2947 punch from 0x7fcbbe8 to 0x7fcc15f, (0x577 bytes) 2024-04-23T23:50:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:2948 read 0x38a50c thru 0x38d05d (0x2b52 bytes) 2024-04-23T23:50:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:2949 punch from 0x99e733 to 0x9a918b, (0xaa58 bytes) 2024-04-23T23:50:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:2950 write 0x58d3a10 thru 0x58e2799 (0xed8a bytes) 2024-04-23T23:50:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:2951 punch from 0x67a28d2 to 0x67a3d49, (0x1477 bytes) 2024-04-23T23:50:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:2952 read 0x16e87de thru 0x16e9540 (0xd63 bytes) 2024-04-23T23:50:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:2953 writesame 0x35847eb thru 0x3591dc8 data_size 0x96(0xd5de bytes) 2024-04-23T23:50:26.383 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:50:26.384 INFO:teuthology.orchestra.run.smithi092.stdout:2954 write 0xa1376f3 thru 0xa13c4e9 (0x4df7 bytes) 2024-04-23T23:50:26.384 INFO:teuthology.orchestra.run.smithi092.stdout:2956 read 0x761632d thru 0x7623267 (0xcf3b bytes) 2024-04-23T23:50:26.384 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.384 INFO:teuthology.orchestra.run.smithi092.stdout:2958 write 0x3551898 thru 0x3561556 (0xfcbf bytes) 2024-04-23T23:50:26.384 INFO:teuthology.orchestra.run.smithi092.stdout:2959 read 0x138ebb6 thru 0x1397623 (0x8a6e bytes) 2024-04-23T23:50:26.385 INFO:teuthology.orchestra.run.smithi092.stdout:2962 writesame 0x937302b thru 0x9375879 data_size 0x1(0x284f bytes) 2024-04-23T23:50:26.391 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.391 INFO:teuthology.orchestra.run.smithi092.stdout:2964 read 0x94ccde0 thru 0x94d5422 (0x8643 bytes) 2024-04-23T23:50:26.391 INFO:teuthology.orchestra.run.smithi092.stdout:2965 writesame 0xe07ddde thru 0xe0895d7 data_size 0x1(0xb7fa bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2966 write 0x794481f thru 0x794dc54 (0x9436 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2968 read 0xc9b35bc thru 0xc9b367a (0xbf bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2970 writesame 0x31f0a6a thru 0x31feb6a data_size 0x1(0xe101 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2971 write 0x6bb9eaa thru 0x6bc1942 (0x7a99 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2973 read 0xdb1349b thru 0xdb18ff5 (0x5b5b bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2974 write 0x60689b9 thru 0x6074d8a (0xc3d2 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2975 read 0x55d1061 thru 0x55e0d06 (0xfca6 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2976 read 0xae85645 thru 0xae8e36c (0x8d28 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2978 trunc from 0xe0895d8 to 0x7ed80cc 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2979 read 0x3bbab2d thru 0x3bc2e8d (0x8361 bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2981 writesame 0xa159beb thru 0xa163f08 data_size 0x1(0xa31e bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2983 punch from 0x288367e to 0x288890d, (0x528f bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:2985 read 0x7e9e703 thru 0x7e9fd9c (0x169a bytes) 2024-04-23T23:50:26.392 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2987 trunc from 0xa163f09 to 0x2ed5e02 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2989 read 0xf2f86a thru 0xf3c183 (0xc91a bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2990 read 0x10ef738 thru 0x10f11e3 (0x1aac bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2993 punch from 0x21527b1 to 0x215fa75, (0xd2c4 bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2995 read 0x1c0343e thru 0x1c0eb99 (0xb75c bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2996 read 0x1f5bccf thru 0x1f671b8 (0xb4ea bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:2997 write 0x90afd0c thru 0x90b5f59 (0x624e bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:3000 writesame 0xc392160 thru 0xc399e4e data_size 0x7(0x7cef bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:3001 write 0x5dd68e1 thru 0x5ddc289 (0x59a9 bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:3003 read 0x34acd08 thru 0x34b96c1 (0xc9ba bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:3004 read 0x42214fc thru 0x422c27c (0xad81 bytes) 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:3005 trunc from 0xc399e4f to 0xacfc484 2024-04-23T23:50:26.393 INFO:teuthology.orchestra.run.smithi092.stdout:3007 read 0x261a004 thru 0x26288bd (0xe8ba bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3008 write 0xba23bcd thru 0xba2e4f0 (0xa924 bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3009 read 0xa01255c thru 0xa01c28a (0x9d2f bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3010 read 0x449b920 thru 0x44ab361 (0xfa42 bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3013 writesame 0x190e3e7 thru 0x1917618 data_size 0x1(0x9232 bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3015 punch from 0x95107b0 to 0x951fb99, (0xf3e9 bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3016 read 0x94e718a thru 0x94f479d (0xd614 bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3017 punch from 0x42f3a91 to 0x42f4400, (0x96f bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3018 punch from 0x761d2a3 to 0x7620a4c, (0x37a9 bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3023 trunc from 0xba2e4f1 to 0x5658817 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:3025 write 0x1affbf3 thru 0x1b07980 (0x7d8e bytes) 2024-04-23T23:50:26.394 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3029 write 0x7b70f6c thru 0x7b73bd8 (0x2c6d bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3030 punch from 0x6080e20 to 0x608a1d8, (0x93b8 bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3031 write 0xb085875 thru 0xb087a42 (0x21ce bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3034 punch from 0x90f2b81 to 0x9101472, (0xe8f1 bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3036 trunc from 0xb087a43 to 0x80a7f91 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3037 write 0x3207666 thru 0x3214cac (0xd647 bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3038 write 0xcd7bf84 thru 0xcd8b4ed (0xf56a bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3039 read 0x68146f5 thru 0x68188ed (0x41f9 bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3040 trunc from 0xcd8b4ee to 0x655923d 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3041 read 0x5c1ff31 thru 0x5c24faf (0x507f bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3042 read 0xf027cc thru 0xf0747b (0x4cb0 bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3043 write 0x6482152 thru 0x6492018 (0xfec7 bytes) 2024-04-23T23:50:26.395 INFO:teuthology.orchestra.run.smithi092.stdout:3044 read 0x64180fb thru 0x6424f29 (0xce2f bytes) 2024-04-23T23:50:26.396 INFO:teuthology.orchestra.run.smithi092.stdout:3045 punch from 0x59b65c8 to 0x59bfbdd, (0x9615 bytes) 2024-04-23T23:50:26.396 INFO:teuthology.orchestra.run.smithi092.stdout:3046 trunc from 0x655923d to 0x46ba03e 2024-04-23T23:50:26.396 INFO:teuthology.orchestra.run.smithi092.stdout:3047 read 0xf3276b thru 0xf3819a (0x5a30 bytes) 2024-04-23T23:50:26.396 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.396 INFO:teuthology.orchestra.run.smithi092.stdout:3049 read 0x15e8806 thru 0x15eb753 (0x2f4e bytes) 2024-04-23T23:50:26.396 INFO:teuthology.orchestra.run.smithi092.stdout:3050 write 0x5ece218 thru 0x5ed5537 (0x7320 bytes) 2024-04-23T23:50:26.396 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:26.389+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:26.397 INFO:teuthology.orchestra.run.smithi092.stdout:3051 writesame 0xab84cfa thru 0xab8a414 data_size 0x1(0x571b bytes) 2024-04-23T23:50:26.397 INFO:teuthology.orchestra.run.smithi092.stdout:3052 write 0xbbfcbe7 thru 0xbbfdbc5 (0xfdf bytes) 2024-04-23T23:50:26.397 INFO:teuthology.orchestra.run.smithi092.stdout:3053 write 0x9369e6f thru 0x9379418 (0xf5aa bytes) 2024-04-23T23:50:26.400 INFO:teuthology.orchestra.run.smithi092.stdout:3054 read 0x3faa539 thru 0x3fba10b (0xfbd3 bytes) 2024-04-23T23:50:26.401 INFO:teuthology.orchestra.run.smithi092.stdout:3055 punch from 0x3762f12 to 0x376edba, (0xbea8 bytes) 2024-04-23T23:50:26.401 INFO:teuthology.orchestra.run.smithi092.stdout:3056 punch from 0xf37586 to 0xf417a8, (0xa222 bytes) 2024-04-23T23:50:26.401 INFO:teuthology.orchestra.run.smithi092.stdout:3057 read 0x9a96b23 thru 0x9a9d55c (0x6a3a bytes) 2024-04-23T23:50:26.408 INFO:teuthology.orchestra.run.smithi092.stdout:3058 trunc from 0xbbfdbc6 to 0x60013db 2024-04-23T23:50:26.419 INFO:teuthology.orchestra.run.smithi092.stdout:3059 writesame 0xb064ac thru 0xb09d0e data_size 0x1(0x3863 bytes) 2024-04-23T23:50:26.446 INFO:teuthology.orchestra.run.smithi092.stdout:3060 write 0xed83adf thru 0xed915c7 (0xdae9 bytes) 2024-04-23T23:50:26.449 INFO:teuthology.orchestra.run.smithi092.stdout:3061 punch from 0xe0525ef to 0xe059333, (0x6d44 bytes) 2024-04-23T23:50:26.449 INFO:teuthology.orchestra.run.smithi092.stdout:3062 punch from 0xe068336 to 0xe072f8a, (0xac54 bytes) 2024-04-23T23:50:26.449 INFO:teuthology.orchestra.run.smithi092.stdout:3065 write 0x552b907 thru 0x5530023 (0x471d bytes) 2024-04-23T23:50:26.453 INFO:teuthology.orchestra.run.smithi092.stdout:3067 write 0x64d1d94 thru 0x64d45f7 (0x2864 bytes) 2024-04-23T23:50:26.456 INFO:teuthology.orchestra.run.smithi092.stdout:3068 writesame 0x4c10e3f thru 0x4c1d5e6 data_size 0x4(0xc7a8 bytes) 2024-04-23T23:50:26.470 INFO:teuthology.orchestra.run.smithi092.stdout:3070 trunc from 0xed915c8 to 0xa4560a3 2024-04-23T23:50:26.476 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:26.490+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:26.482 INFO:teuthology.orchestra.run.smithi092.stdout:3071 write 0x570b8c thru 0x573791 (0x2c06 bytes) 2024-04-23T23:50:26.484 INFO:teuthology.orchestra.run.smithi092.stdout:3072 writesame 0x7b08960 thru 0x7b12cf5 data_size 0x1(0xa396 bytes) 2024-04-23T23:50:26.485 INFO:teuthology.orchestra.run.smithi092.stdout:3073 read 0x371c23d thru 0x372a984 (0xe748 bytes) 2024-04-23T23:50:26.486 INFO:teuthology.orchestra.run.smithi092.stdout:3075 punch from 0x2d334cd to 0x2d369bf, (0x34f2 bytes) 2024-04-23T23:50:26.486 INFO:teuthology.orchestra.run.smithi092.stdout:3076 punch from 0x9a3f25 to 0x9aa58f, (0x666a bytes) 2024-04-23T23:50:26.486 INFO:teuthology.orchestra.run.smithi092.stdout:3078 write 0x1acd68f thru 0x1acd6dc (0x4e bytes) 2024-04-23T23:50:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:3079 writesame 0x37b2120 thru 0x37b31a8 data_size 0x11(0x1089 bytes) 2024-04-23T23:50:26.492 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:26.492 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:26.492 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:26.492 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:26.493 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:26.500 INFO:teuthology.orchestra.run.smithi092.stdout:3081 trunc from 0xa4560a3 to 0x4b63a3 2024-04-23T23:50:26.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:26.516 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.523 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-04-23T23:50:26.523 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:26.523 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-04-23T23:50:26.524 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-23T23:50:26.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-23T23:50:26.526 INFO:tasks.ceph.osd.3:Started 2024-04-23T23:50:26.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:26.527 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:26.527 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:26.536 INFO:teuthology.orchestra.run.smithi092.stdout:3083 writesame 0xd29364f thru 0xd2a2a9f data_size 0x7(0xf451 bytes) 2024-04-23T23:50:26.536 INFO:teuthology.orchestra.run.smithi092.stdout:3084 read 0x8d06894 thru 0x8d096a8 (0x2e15 bytes) 2024-04-23T23:50:26.543 INFO:teuthology.orchestra.run.smithi092.stdout:3087 trunc from 0xd2a2aa0 to 0x79000e2 2024-04-23T23:50:26.552 INFO:teuthology.orchestra.run.smithi092.stdout:3089 trunc from 0x79000e2 to 0xb420f2a 2024-04-23T23:50:26.553 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:26.553 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:26.558 INFO:teuthology.orchestra.run.smithi092.stdout:3091 writesame 0x542d30 thru 0x551f92 data_size 0x1(0xf263 bytes) 2024-04-23T23:50:26.571 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:26.572 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:26.572 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:26.578 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:26.591+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:26.594 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:26.594 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:26.605 INFO:teuthology.orchestra.run.smithi092.stdout:3093 punch from 0x3a69bff to 0x3a77e00, (0xe201 bytes) 2024-04-23T23:50:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:3094 trunc from 0xb420f2a to 0x8cd4d1c 2024-04-23T23:50:26.610 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:26.610 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36195 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1849344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.351138+0000", 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2726289839999998, 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047635104999999997, 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36195 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351138+0000", 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351138+0000", 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351138+0000", 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0600000000000001e-07 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351140+0000", 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.052e-06 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351146+0000", 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1310000000000003e-06 2024-04-23T23:50:26.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351151+0000", 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4730000000000002e-06 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.398634+0000", 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047483105999999997 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.398697+0000", 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3328000000000001e-05 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.398773+0000", 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5609000000000001e-05 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36196 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1853440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.351157+0000", 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.272609895, 2024-04-23T23:50:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047876876999999998, 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36196 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351157+0000", 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351157+0000", 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351157+0000", 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.08e-07 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351159+0000", 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0559999999999999e-06 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351165+0000", 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9039999999999997e-06 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351170+0000", 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.398818+0000", 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047647215999999999 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.398907+0000", 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9220000000000003e-05 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.399034+0000", 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012677700000000001 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.619 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36197 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1857536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.351176+0000", 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.272590246, 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048106231999999999, 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36197 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351176+0000", 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.351176+0000", 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352828+0000", 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016513280000000001 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352832+0000", 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6009999999999997e-06 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352841+0000", 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8200000000000003e-06 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352846+0000", 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.989e-06 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.399077+0000", 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.046230347999999997 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.399155+0000", 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8209999999999998e-05 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.399283+0000", 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012793599999999999 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36203 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1882112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.352981+0000", 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2707855769999998, 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047600890999999999, 2024-04-23T23:50:26.624 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36203 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352981+0000", 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352981+0000", 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352982+0000", 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1699999999999998e-07 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352984+0000", 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1179999999999999e-06 2024-04-23T23:50:26.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352990+0000", 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5019999999999998e-06 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.352996+0000", 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8289999999999996e-06 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400475+0000", 2024-04-23T23:50:26.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047479210000000001 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400517+0000", 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1548999999999998e-05 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400582+0000", 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5166000000000006e-05 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36204 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.353003+0000", 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2707640590000002, 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047717467999999999, 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36204 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.629 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353003+0000", 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353003+0000", 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353003+0000", 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353005+0000", 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9010000000000001e-06 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353011+0000", 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0059999999999998e-06 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353016+0000", 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8369999999999996e-06 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400602+0000", 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047585972999999997 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400652+0000", 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9811000000000001e-05 2024-04-23T23:50:26.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400720+0000", 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8496000000000003e-05 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36205 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1890304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.353022+0000", 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2707442019999999, 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047869619000000002, 2024-04-23T23:50:26.633 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36205 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353022+0000", 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353022+0000", 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353023+0000", 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0200000000000003e-07 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353025+0000", 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3649999999999998e-06 2024-04-23T23:50:26.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353032+0000", 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.46e-06 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353038+0000", 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8710000000000002e-06 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400747+0000", 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047709164999999998 2024-04-23T23:50:26.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400788+0000", 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1010000000000002e-05 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400892+0000", 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010434599999999999 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36206 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1894400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.353044+0000", 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2707227290000001, 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048076555999999999, 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36206 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.638 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353044+0000", 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353044+0000", 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353044+0000", 2024-04-23T23:50:26.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2300000000000002e-07 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353046+0000", 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1069999999999999e-06 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353053+0000", 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.336e-06 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353058+0000", 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0919999999999997e-06 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400928+0000", 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047869805000000001 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.400999+0000", 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1579e-05 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.401121+0000", 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121214 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36208 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1898496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.353089+0000", 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2706781629999999, 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048272109000000001, 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.642 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36208 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353089+0000", 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353089+0000", 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353089+0000", 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9100000000000004e-07 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353091+0000", 2024-04-23T23:50:26.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.148e-06 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353096+0000", 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1490000000000002e-06 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353099+0000", 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.05e-06 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.401169+0000", 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048069959000000002 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.401241+0000", 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1716000000000003e-05 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.401361+0000", 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119496 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36210 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1902592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.353116+0000", 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2706501860000001, 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048912867999999998, 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36210 2024-04-23T23:50:26.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353116+0000", 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353116+0000", 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353117+0000", 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0600000000000001e-07 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353118+0000", 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.257e-06 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353122+0000", 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6370000000000001e-06 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353125+0000", 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8389999999999998e-06 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.401391+0000", 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048265984999999997 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.401462+0000", 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1923999999999998e-05 2024-04-23T23:50:26.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402029+0000", 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056692000000000005 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36212 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.651 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.353139+0000", 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2706277570000002, 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049134931, 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36212 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353139+0000", 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.353139+0000", 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354848+0000", 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017087910000000001 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354854+0000", 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4300000000000003e-06 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354861+0000", 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1609999999999997e-06 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354866+0000", 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402068+0000", 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047202417000000003 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402156+0000", 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7876999999999999e-05 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402274+0000", 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117724 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36214 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1910784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.656 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.354891+0000", 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2688757819999998, 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047616369999999998, 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36214 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354891+0000", 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354891+0000", 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354891+0000", 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6e-07 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354893+0000", 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3769999999999999e-06 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354896+0000", 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3409999999999999e-06 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354899+0000", 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.791e-06 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402318+0000", 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047419280000000001 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402392+0000", 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4105999999999998e-05 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402507+0000", 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115215 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36217 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1914880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.354938+0000", 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.268828837, 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047755151000000003, 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36217 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354938+0000", 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354938+0000", 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354938+0000", 2024-04-23T23:50:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9999999999999998e-07 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354940+0000", 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8029999999999999e-06 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354945+0000", 2024-04-23T23:50:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3809999999999998e-06 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354950+0000", 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4360000000000004e-06 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402528+0000", 2024-04-23T23:50:26.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047578068000000001 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402587+0000", 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9530000000000001e-05 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402693+0000", 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010553299999999999 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36219 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.354973+0000", 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2687938189999999, 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047958552000000002, 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.668 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.669 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.669 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.669 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.669 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.669 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36219 2024-04-23T23:50:26.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354973+0000", 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354973+0000", 2024-04-23T23:50:26.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354973+0000", 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5100000000000001e-07 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354974+0000", 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.299e-06 2024-04-23T23:50:26.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354978+0000", 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2739999999999999e-06 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354980+0000", 2024-04-23T23:50:26.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7209999999999999e-06 2024-04-23T23:50:26.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402731+0000", 2024-04-23T23:50:26.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047750810999999997 2024-04-23T23:50:26.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402804+0000", 2024-04-23T23:50:26.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3005999999999999e-05 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402931+0000", 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012719000000000001 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36221 3.2 3:407fe0cc:::rbd_data.11796b6600d0.0000000000000003:15 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.354993+0000", 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2687731759999998, 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048194962000000001, 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36221 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.677 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354993+0000", 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354993+0000", 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354994+0000", 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1199999999999999e-07 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354995+0000", 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.203e-06 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.354998+0000", 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9270000000000001e-06 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355000+0000", 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5280000000000002e-06 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.402977+0000", 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0479763 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.403069+0000", 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2403000000000003e-05 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.403188+0000", 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119389 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36223 3.2 3.3307fe02 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.355018+0000", 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.268749068, 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04821748, 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36223 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355018+0000", 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355018+0000", 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355018+0000", 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5400000000000002e-07 2024-04-23T23:50:26.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355019+0000", 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.28e-06 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355022+0000", 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2669999999999998e-06 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355025+0000", 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6620000000000001e-06 2024-04-23T23:50:26.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.403225+0000", 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048199998000000001 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.403235+0000", 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0019e-05 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.685 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:36224 3.2 3.3307fe02 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.355028+0000", 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.2687383379999999, 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048230589999999997, 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 36224 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355028+0000", 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.355028+0000", 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.356830+0000", 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018012 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.356836+0000", 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4010000000000002e-06 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.356845+0000", 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1749999999999994e-06 2024-04-23T23:50:26.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.356850+0000", 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8949999999999997e-06 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.403250+0000", 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.046400236999999997 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.403259+0000", 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6819999999999992e-06 2024-04-23T23:50:26.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:45537 3.5 3:a03bd1e7:::rbd_data.11796b6600d0.0000000000000007:15 [read 2082816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.183261+0000", 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.440506115, 2024-04-23T23:50:26.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047675909000000002, 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 45537 2024-04-23T23:50:26.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183261+0000", 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183261+0000", 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183261+0000", 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2399999999999999e-07 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183262+0000", 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1319999999999999e-06 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183266+0000", 2024-04-23T23:50:26.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9149999999999998e-06 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183269+0000", 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.774e-06 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.230820+0000", 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047550907000000003 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.230867+0000", 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7324999999999998e-05 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.230936+0000", 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9632e-05 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:45538 3.5 3:a03bd1e7:::rbd_data.11796b6600d0.0000000000000007:15 [read 2086912~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.183272+0000", 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.440494776, 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047804966999999997, 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 45538 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183272+0000", 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183272+0000", 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183272+0000", 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.23e-07 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183276+0000", 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4010000000000001e-06 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183280+0000", 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0779999999999997e-06 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183283+0000", 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0939999999999999e-06 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.230963+0000", 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047679998000000001 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231006+0000", 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2908999999999999e-05 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231077+0000", 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1264000000000001e-05 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:45539 3.5 3:a03bd1e7:::rbd_data.11796b6600d0.0000000000000007:15 [read 2091008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.183286+0000", 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.440480816, 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047978719000000003, 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 45539 2024-04-23T23:50:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183286+0000", 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183286+0000", 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183286+0000", 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2999999999999999e-07 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183287+0000", 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.142e-06 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183297+0000", 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2839999999999995e-06 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183300+0000", 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6090000000000001e-06 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231117+0000", 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047816918999999999 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231186+0000", 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9277999999999996e-05 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231265+0000", 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8257000000000006e-05 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:45540 3.5 3:a03bd1e7:::rbd_data.11796b6600d0.0000000000000007:15 [read 2095104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.183304+0000", 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.440463112, 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048098104000000003, 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:26.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 45540 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183304+0000", 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:26.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183304+0000", 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183304+0000", 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6199999999999999e-07 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183305+0000", 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.238e-06 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:26.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183308+0000", 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2200000000000001e-06 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.183311+0000", 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5969999999999999e-06 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231287+0000", 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047975971999999999 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:26.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231331+0000", 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4535e-05 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.231402+0000", 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0279999999999998e-05 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:26.713 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:26.714 INFO:teuthology.orchestra.run.smithi092.stdout:3098 write 0x65439a8 thru 0x6549e56 (0x64af bytes) 2024-04-23T23:50:26.715 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:26.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:26.717 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:26.728+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:26.717 INFO:teuthology.orchestra.run.smithi092.stdout:3099 writesame 0xe77531e thru 0xe777edc data_size 0x1(0x2bbf bytes) 2024-04-23T23:50:26.717 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.717 INFO:teuthology.orchestra.run.smithi092.stdout:3102 write 0x4bc9d00 thru 0x4bd8720 (0xea21 bytes) 2024-04-23T23:50:26.717 INFO:teuthology.orchestra.run.smithi092.stdout:3103 read 0x39da214 thru 0x39e5259 (0xb046 bytes) 2024-04-23T23:50:26.717 INFO:teuthology.orchestra.run.smithi092.stdout:3104 write 0x8d7b117 thru 0x8d86f63 (0xbe4d bytes) 2024-04-23T23:50:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:26.718 INFO:teuthology.orchestra.run.smithi092.stdout:3105 write 0xe6e6066 thru 0xe6f1f56 (0xbef1 bytes) 2024-04-23T23:50:26.719 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:config diff 2024-04-23T23:50:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:config diff get 2024-04-23T23:50:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:config get 2024-04-23T23:50:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:config help [] 2024-04-23T23:50:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:config set ... 2024-04-23T23:50:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:admin_socket: invalid command 2024-04-23T23:50:26.720 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-04-23T23:50:26.721 INFO:teuthology.orchestra.run.smithi092.stdout:3106 write 0x5853ae thru 0x586e67 (0x1aba bytes) 2024-04-23T23:50:26.721 INFO:teuthology.orchestra.run.smithi092.stdout:3108 read 0xa045d65 thru 0xa04c2ea (0x6586 bytes) 2024-04-23T23:50:26.721 INFO:teuthology.orchestra.run.smithi092.stdout:3109 write 0xa77c224 thru 0xa7868b1 (0xa68e bytes) 2024-04-23T23:50:26.721 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:26.722 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:26.722 INFO:teuthology.orchestra.run.smithi092.stdout:3111 trunc from 0xe777edd to 0x33e1c76 2024-04-23T23:50:26.722 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:26.723 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:26.723 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:26.724 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3113 writesame 0x670b3f7 thru 0x6711674 data_size 0x2(0x627e bytes) 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3114 trunc from 0x6711675 to 0x9628e58 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3115 punch from 0x21add0 to 0x228f99, (0xe1c9 bytes) 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3117 punch from 0x8e18807 to 0x8e207a8, (0x7fa1 bytes) 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3118 writesame 0x172cfd7 thru 0x173637b data_size 0x2b(0x93a5 bytes) 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3119 punch from 0xe6bd44 to 0xe6c4f9, (0x7b5 bytes) 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3120 read 0x89dad4e thru 0x89e5502 (0xa7b5 bytes) 2024-04-23T23:50:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:3122 writesame 0xd914bc3 thru 0xd9189c4 data_size 0x1(0x3e02 bytes) 2024-04-23T23:50:26.726 INFO:teuthology.orchestra.run.smithi092.stdout:3123 trunc from 0xd9189c5 to 0x4ae1c4e 2024-04-23T23:50:26.726 INFO:teuthology.orchestra.run.smithi092.stdout:3124 punch from 0xd9cb73 to 0xd9f1e0, (0x266d bytes) 2024-04-23T23:50:26.726 INFO:teuthology.orchestra.run.smithi092.stdout:3127 punch from 0x43979e6 to 0x4399c5c, (0x2276 bytes) 2024-04-23T23:50:26.726 INFO:teuthology.orchestra.run.smithi092.stdout:3128 read 0x20acc3a thru 0x20ad190 (0x557 bytes) 2024-04-23T23:50:26.732 INFO:teuthology.orchestra.run.smithi092.stdout:3129 write 0xc57c2f6 thru 0xc58b5d0 (0xf2db bytes) 2024-04-23T23:50:26.735 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.736 INFO:teuthology.orchestra.run.smithi092.stdout:3132 trunc from 0xc58b5d1 to 0xd5ac981 2024-04-23T23:50:26.742 INFO:teuthology.orchestra.run.smithi092.stdout:3133 write 0xcc4a738 thru 0xcc585ff (0xdec8 bytes) 2024-04-23T23:50:26.746 INFO:teuthology.orchestra.run.smithi092.stdout:3135 punch from 0xb76c7d9 to 0xb76e59b, (0x1dc2 bytes) 2024-04-23T23:50:26.746 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.748 INFO:teuthology.orchestra.run.smithi092.stdout:3138 trunc from 0xd5ac981 to 0xb1d59ca 2024-04-23T23:50:26.768 INFO:teuthology.orchestra.run.smithi092.stdout:3139 write 0xbf50e9a thru 0xbf602af (0xf416 bytes) 2024-04-23T23:50:26.771 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.772 INFO:teuthology.orchestra.run.smithi092.stdout:3142 write 0x8ce6083 thru 0x8cf0e6b (0xade9 bytes) 2024-04-23T23:50:26.785 INFO:teuthology.orchestra.run.smithi092.stdout:3143 trunc from 0xbf602b0 to 0x34016d9 2024-04-23T23:50:26.795 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.804 INFO:teuthology.orchestra.run.smithi092.stdout:3146 trunc from 0x34016d9 to 0xb32cee5 2024-04-23T23:50:26.810 INFO:teuthology.orchestra.run.smithi092.stdout:3149 write 0x9f0b03 thru 0x9f0e2a (0x328 bytes) 2024-04-23T23:50:26.814 INFO:teuthology.orchestra.run.smithi092.stdout:3150 writesame 0x15be76c thru 0x15ce4bf data_size 0x3e(0xfd54 bytes) 2024-04-23T23:50:26.815 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:26.829+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:26.816 INFO:teuthology.orchestra.run.smithi092.stdout:3153 write 0x5ad98ca thru 0x5ae2949 (0x9080 bytes) 2024-04-23T23:50:26.832 INFO:teuthology.orchestra.run.smithi092.stdout:3154 trunc from 0xb32cee5 to 0xa7fba3 2024-04-23T23:50:26.865 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:26.940 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:26.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:26.943 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:26.956+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:26.944 INFO:teuthology.orchestra.run.smithi092.stdout:3155 write 0xc01f980 thru 0xc02bce4 (0xc365 bytes) 2024-04-23T23:50:26.944 INFO:teuthology.orchestra.run.smithi092.stdout:3156 read 0x2dd8c3b thru 0x2de780e (0xebd4 bytes) 2024-04-23T23:50:26.944 INFO:teuthology.orchestra.run.smithi092.stdout:3159 writesame 0xdda6c60 thru 0xddad98b data_size 0x22(0x6d2c bytes) 2024-04-23T23:50:26.944 INFO:teuthology.orchestra.run.smithi092.stdout:3160 writesame 0x4f7bed thru 0x502e73 data_size 0x1(0xb287 bytes) 2024-04-23T23:50:26.944 INFO:teuthology.orchestra.run.smithi092.stdout:3161 write 0xd03abdc thru 0xd043df7 (0x921c bytes) 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:3162 writesame 0x386a944 thru 0x3878c11 data_size 0x3(0xe2ce bytes) 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:3163 trunc from 0xddad98c to 0xec22d2b 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:3164 punch from 0xab99f01 to 0xaba91b8, (0xf2b7 bytes) 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:3166 punch from 0x137484c to 0x13842c9, (0xfa7d bytes) 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:3167 write 0x846e53 thru 0x8504cc (0x967a bytes) 2024-04-23T23:50:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:3168 read 0xab2488a thru 0xab261e7 (0x195e bytes) 2024-04-23T23:50:26.946 INFO:teuthology.orchestra.run.smithi092.stdout:3171 write 0x497ebf5 thru 0x498269d (0x3aa9 bytes) 2024-04-23T23:50:26.946 INFO:teuthology.orchestra.run.smithi092.stdout:3173 trunc from 0xec22d2b to 0x3533e74 2024-04-23T23:50:26.972 INFO:teuthology.orchestra.run.smithi092.stdout:3174 write 0xcffa96f thru 0xd009b4f (0xf1e1 bytes) 2024-04-23T23:50:26.975 INFO:teuthology.orchestra.run.smithi092.stdout:3176 punch from 0xb478294 to 0xb47d2de, (0x504a bytes) 2024-04-23T23:50:26.975 INFO:teuthology.orchestra.run.smithi092.stdout:3177 write 0x8fd7bc0 thru 0x8fdc26b (0x46ac bytes) 2024-04-23T23:50:26.978 INFO:teuthology.orchestra.run.smithi092.stdout:3178 writesame 0x6d765f1 thru 0x6d851ef data_size 0x5(0xebff bytes) 2024-04-23T23:50:26.986 INFO:teuthology.orchestra.run.smithi092.stdout:3179 read 0x914348b thru 0x914de94 (0xaa0a bytes) 2024-04-23T23:50:26.986 INFO:teuthology.orchestra.run.smithi092.stdout:3180 write 0x19fed5f thru 0x1a05f28 (0x71ca bytes) 2024-04-23T23:50:26.997 INFO:teuthology.orchestra.run.smithi092.stdout:3181 trunc from 0xd009b50 to 0x45f98e0 2024-04-23T23:50:27.012 INFO:teuthology.orchestra.run.smithi092.stdout:3182 read 0xfaa050 thru 0xfb0122 (0x60d3 bytes) 2024-04-23T23:50:27.030 INFO:teuthology.orchestra.run.smithi092.stdout:3185 write 0xdea0a62 thru 0xdeae737 (0xdcd6 bytes) 2024-04-23T23:50:27.033 INFO:teuthology.orchestra.run.smithi092.stdout:3186 read 0x19712e0 thru 0x197fb6f (0xe890 bytes) 2024-04-23T23:50:27.034 INFO:teuthology.orchestra.run.smithi092.stdout:3190 read 0xb0db96f thru 0xb0e7f78 (0xc60a bytes) 2024-04-23T23:50:27.034 INFO:teuthology.orchestra.run.smithi092.stdout:3191 write 0xcab0a23 thru 0xcab7796 (0x6d74 bytes) 2024-04-23T23:50:27.037 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.037 INFO:teuthology.orchestra.run.smithi092.stdout:3194 write 0x1c7c265 thru 0x1c8b08a (0xee26 bytes) 2024-04-23T23:50:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:3195 read 0xc05d488 thru 0xc064429 (0x6fa2 bytes) 2024-04-23T23:50:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:3197 read 0xd8b4fab thru 0xd8bb37e (0x63d4 bytes) 2024-04-23T23:50:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:3198 punch from 0x15eaac9 to 0x15fa436, (0xf96d bytes) 2024-04-23T23:50:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:3200 punch from 0xc914b5c to 0xc914ee3, (0x387 bytes) 2024-04-23T23:50:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:3201 write 0x1b693c thru 0x1beb37 (0x81fc bytes) 2024-04-23T23:50:27.040 INFO:teuthology.orchestra.run.smithi092.stdout:3203 read 0x1572617 thru 0x1581128 (0xeb12 bytes) 2024-04-23T23:50:27.042 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:27.055+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:27.054 INFO:teuthology.orchestra.run.smithi092.stdout:3206 trunc from 0xdeae738 to 0x2ea0458 2024-04-23T23:50:27.068 INFO:teuthology.orchestra.run.smithi092.stdout:3208 read 0xdeaafd thru 0xdef2c0 (0x47c4 bytes) 2024-04-23T23:50:27.075 INFO:teuthology.orchestra.run.smithi092.stdout:3209 trunc from 0x2ea0458 to 0x8da1df0 2024-04-23T23:50:27.080 INFO:teuthology.orchestra.run.smithi092.stdout:3210 write 0x22c8446 thru 0x22d1f62 (0x9b1d bytes) 2024-04-23T23:50:27.082 INFO:teuthology.orchestra.run.smithi092.stdout:3211 punch from 0x55eb892 to 0x55ee8ed, (0x305b bytes) 2024-04-23T23:50:27.082 INFO:teuthology.orchestra.run.smithi092.stdout:3212 write 0x18f2234 thru 0x18fd532 (0xb2ff bytes) 2024-04-23T23:50:27.083 INFO:teuthology.orchestra.run.smithi092.stdout:3213 writesame 0x4f926d6 thru 0x4f95085 data_size 0x2e(0x29b0 bytes) 2024-04-23T23:50:27.086 INFO:teuthology.orchestra.run.smithi092.stdout:3214 writesame 0x71dee90 thru 0x71ee404 data_size 0x1(0xf575 bytes) 2024-04-23T23:50:27.092 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.092 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:27.121 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.121 INFO:teuthology.orchestra.run.smithi092.stdout:3218 writesame 0x6309d81 thru 0x63165c7 data_size 0xb(0xc847 bytes) 2024-04-23T23:50:27.135 INFO:teuthology.orchestra.run.smithi092.stdout:3219 trunc from 0x8da1df0 to 0xeae1a32 2024-04-23T23:50:27.140 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:27.153+0000 7efdcc218640 -1 Falling back to public interface 2024-04-23T23:50:27.141 INFO:teuthology.orchestra.run.smithi092.stdout:3220 writesame 0xdf00c78 thru 0xdf0c9fd data_size 0x11(0xbd86 bytes) 2024-04-23T23:50:27.145 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:27.159+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:27.146 INFO:teuthology.orchestra.run.smithi092.stdout:3221 read 0xd61a605 thru 0xd626460 (0xbe5c bytes) 2024-04-23T23:50:27.147 INFO:teuthology.orchestra.run.smithi092.stdout:3223 read 0x8af940b thru 0x8b02b41 (0x9737 bytes) 2024-04-23T23:50:27.147 INFO:teuthology.orchestra.run.smithi092.stdout:3224 read 0xb01f351 thru 0xb02d081 (0xdd31 bytes) 2024-04-23T23:50:27.147 INFO:teuthology.orchestra.run.smithi092.stdout:3225 write 0xbc2d577 thru 0xbc39756 (0xc1e0 bytes) 2024-04-23T23:50:27.148 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.149 INFO:teuthology.orchestra.run.smithi092.stdout:3227 read 0x5fea1b9 thru 0x5ff6b3d (0xc985 bytes) 2024-04-23T23:50:27.149 INFO:teuthology.orchestra.run.smithi092.stdout:3230 punch from 0x67a31b3 to 0x67b0e3f, (0xdc8c bytes) 2024-04-23T23:50:27.155 INFO:teuthology.orchestra.run.smithi092.stdout:3231 trunc from 0xeae1a32 to 0x9352c06 2024-04-23T23:50:27.178 INFO:teuthology.orchestra.run.smithi092.stdout:3232 read 0x61e38b3 thru 0x61e7457 (0x3ba5 bytes) 2024-04-23T23:50:27.191 INFO:teuthology.orchestra.run.smithi092.stdout:3234 write 0xdf1cb43 thru 0xdf27f4e (0xb40c bytes) 2024-04-23T23:50:27.194 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.194 INFO:teuthology.orchestra.run.smithi092.stdout:3237 writesame 0x540a885 thru 0x540ff0f data_size 0x23(0x568b bytes) 2024-04-23T23:50:27.196 INFO:teuthology.orchestra.run.smithi092.stdout:3239 punch from 0x5f5850f to 0x5f58d29, (0x81a bytes) 2024-04-23T23:50:27.196 INFO:teuthology.orchestra.run.smithi092.stdout:3240 read 0xa360814 thru 0xa369af4 (0x92e1 bytes) 2024-04-23T23:50:27.196 INFO:teuthology.orchestra.run.smithi092.stdout:3241 read 0x91dfcb9 thru 0x91e92c0 (0x9608 bytes) 2024-04-23T23:50:27.196 INFO:teuthology.orchestra.run.smithi092.stdout:3242 write 0x7f798fc thru 0x7f81250 (0x7955 bytes) 2024-04-23T23:50:27.200 INFO:teuthology.orchestra.run.smithi092.stdout:3243 punch from 0x9f0cec2 to 0x9f18897, (0xb9d5 bytes) 2024-04-23T23:50:27.200 INFO:teuthology.orchestra.run.smithi092.stdout:3244 writesame 0xbeffcea thru 0xbf00445 data_size 0x6(0x75c bytes) 2024-04-23T23:50:27.208 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.209 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.213 INFO:teuthology.orchestra.run.smithi092.stdout:3245 trunc from 0xdf27f4f to 0x4c2095f 2024-04-23T23:50:27.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:27.235 INFO:teuthology.orchestra.run.smithi092.stdout:3246 trunc from 0x4c2095f to 0x54437ce 2024-04-23T23:50:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:3248 trunc from 0x54437ce to 0x68a1db0 2024-04-23T23:50:27.251 INFO:teuthology.orchestra.run.smithi092.stdout:3249 read 0x2ece309 thru 0x2edb682 (0xd37a bytes) 2024-04-23T23:50:27.251 INFO:teuthology.orchestra.run.smithi092.stdout:3250 write 0x1535c71 thru 0x1542c55 (0xcfe5 bytes) 2024-04-23T23:50:27.252 INFO:teuthology.orchestra.run.smithi092.stdout:3251 writesame 0x4e211d8 thru 0x4e230a0 data_size 0x25(0x1ec9 bytes) 2024-04-23T23:50:27.259 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:27.273+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:27.269 INFO:teuthology.orchestra.run.smithi092.stdout:3252 write 0xe6e3cef thru 0xe6ec332 (0x8644 bytes) 2024-04-23T23:50:27.284 INFO:teuthology.orchestra.run.smithi092.stdout:3253 trunc from 0xe6ec333 to 0x3a5df11 2024-04-23T23:50:27.294 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.298 INFO:teuthology.orchestra.run.smithi092.stdout:3255 trunc from 0x3a5df11 to 0x722ba47 2024-04-23T23:50:27.303 INFO:teuthology.orchestra.run.smithi092.stdout:3256 read 0x8e10ab thru 0x8ea2b7 (0x920d bytes) 2024-04-23T23:50:27.303 INFO:teuthology.orchestra.run.smithi092.stdout:3257 writesame 0x3da56ec thru 0x3da8226 data_size 0x1f(0x2b3b bytes) 2024-04-23T23:50:27.305 INFO:teuthology.orchestra.run.smithi092.stdout:3258 read 0x1281c8e thru 0x1284224 (0x2597 bytes) 2024-04-23T23:50:27.306 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.306 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.306 INFO:teuthology.orchestra.run.smithi092.stdout:3261 write 0x68e15a7 thru 0x68e58b9 (0x4313 bytes) 2024-04-23T23:50:27.308 INFO:teuthology.orchestra.run.smithi092.stdout:3264 read 0x1324e8f thru 0x1324fac (0x11e bytes) 2024-04-23T23:50:27.309 INFO:teuthology.orchestra.run.smithi092.stdout:3265 trunc from 0x722ba47 to 0x79ab6e0 2024-04-23T23:50:27.320 INFO:teuthology.orchestra.run.smithi092.stdout:3266 write 0x7daa82b thru 0x7dba646 (0xfe1c bytes) 2024-04-23T23:50:27.323 INFO:teuthology.orchestra.run.smithi092.stdout:3268 read 0x5b4b7bc thru 0x5b4f031 (0x3876 bytes) 2024-04-23T23:50:27.330 INFO:teuthology.orchestra.run.smithi092.stdout:3269 write 0x8ce0acb thru 0x8ced5bc (0xcaf2 bytes) 2024-04-23T23:50:27.333 INFO:teuthology.orchestra.run.smithi092.stdout:3271 read 0x20489d8 thru 0x2049c54 (0x127d bytes) 2024-04-23T23:50:27.333 INFO:teuthology.orchestra.run.smithi092.stdout:3272 read 0x672898a thru 0x672d1fe (0x4875 bytes) 2024-04-23T23:50:27.338 INFO:teuthology.orchestra.run.smithi092.stdout:3273 trunc from 0x8ced5bd to 0xcd4a2ed 2024-04-23T23:50:27.344 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:27.358+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:27.352 INFO:teuthology.orchestra.run.smithi092.stdout:3274 write 0xe0f3933 thru 0xe0fe399 (0xaa67 bytes) 2024-04-23T23:50:27.362 INFO:teuthology.orchestra.run.smithi092.stdout:3275 write 0xedb8a21 thru 0xedc1afe (0x90de bytes) 2024-04-23T23:50:27.365 INFO:teuthology.orchestra.run.smithi092.stdout:3276 read 0x872354a thru 0x872cf5c (0x9a13 bytes) 2024-04-23T23:50:27.365 INFO:teuthology.orchestra.run.smithi092.stdout:3278 read 0xe23d27c thru 0xe245775 (0x84fa bytes) 2024-04-23T23:50:27.366 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:27.366 INFO:teuthology.orchestra.run.smithi092.stdout:3280 write 0x569ebc7 thru 0x56a46d0 (0x5b0a bytes) 2024-04-23T23:50:27.370 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.370 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:27.384 INFO:teuthology.orchestra.run.smithi092.stdout:3281 trunc from 0xedc1aff to 0x20379e7 2024-04-23T23:50:27.395 INFO:teuthology.orchestra.run.smithi092.stdout:3282 read 0x10ac8f0 thru 0x10bc2b4 (0xf9c5 bytes) 2024-04-23T23:50:27.396 INFO:teuthology.orchestra.run.smithi092.stdout:3284 punch from 0x2fde73 to 0x30c4f6, (0xe683 bytes) 2024-04-23T23:50:27.412 INFO:teuthology.orchestra.run.smithi092.stdout:3285 write 0xb4cfa62 thru 0xb4d94a3 (0x9a42 bytes) 2024-04-23T23:50:27.415 INFO:teuthology.orchestra.run.smithi092.stdout:3287 clone 21 order 18 su 8192 sc 9 2024-04-23T23:50:27.446 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:27.460+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:27.574 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:27.588+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:27.631 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.631 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:27.684 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:27.698+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:27.744 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.745 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.749 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:27.763+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:27.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:27.850 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:27.864+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:27.895 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:27.895 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:27.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:27.950 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:27.964+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:28.007 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:28.007 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:28.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:28.052 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:28.066+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:28.149 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone20 from 0xb4d94a4 (overlap 0x4b63a3) to 0x0 2024-04-23T23:50:28.153 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:28.166+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:28.165 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:28.165 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:28.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:28.210 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-04-23T23:50:28.253 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:28.267+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:28.355 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:28.369+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:28.361 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:28.361 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:28.366 INFO:teuthology.orchestra.run.smithi092.stdout:3290 write 0xc4c2581 thru 0xc4cfb57 (0xd5d7 bytes) 2024-04-23T23:50:28.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:28.381 INFO:teuthology.orchestra.run.smithi092.stdout:3292 read 0xa7027ab thru 0xa708637 (0x5e8d bytes) 2024-04-23T23:50:28.382 INFO:teuthology.orchestra.run.smithi092.stdout:3294 punch from 0x4d3e88d to 0x4d484f9, (0x9c6c bytes) 2024-04-23T23:50:28.382 INFO:teuthology.orchestra.run.smithi092.stdout:3295 read 0x81bb3de thru 0x81c0c9c (0x58bf bytes) 2024-04-23T23:50:28.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:50:28.394 INFO:teuthology.orchestra.run.smithi092.stdout:3296 trunc from 0xc4cfb58 to 0x10e2ce5 2024-04-23T23:50:28.455 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:28.469+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:28.459 INFO:teuthology.orchestra.run.smithi092.stdout:3297 punch from 0x41fd95 to 0x4288a3, (0x8b0e bytes) 2024-04-23T23:50:28.459 INFO:teuthology.orchestra.run.smithi092.stdout:3298 read 0x45003f thru 0x453744 (0x3706 bytes) 2024-04-23T23:50:28.460 INFO:teuthology.orchestra.run.smithi092.stdout:3304 read 0x32f62 thru 0x41560 (0xe5ff bytes) 2024-04-23T23:50:28.462 INFO:teuthology.orchestra.run.smithi092.stdout:3306 read 0xd39cae thru 0xd3d99a (0x3ced bytes) 2024-04-23T23:50:28.483 INFO:teuthology.orchestra.run.smithi092.stdout:3308 writesame 0xde265b3 thru 0xde2ad91 data_size 0x1(0x47df bytes) 2024-04-23T23:50:28.484 INFO:teuthology.orchestra.run.smithi092.stdout:3309 write 0x2844914 thru 0x2846821 (0x1f0e bytes) 2024-04-23T23:50:28.494 INFO:teuthology.orchestra.run.smithi092.stdout:3310 trunc from 0xde2ad92 to 0x7c74203 2024-04-23T23:50:28.501 INFO:teuthology.orchestra.run.smithi092.stdout:3312 trunc from 0x7c74203 to 0x722b0db 2024-04-23T23:50:28.501 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:28.501 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:28.506 INFO:teuthology.orchestra.run.smithi092.stdout:3314 read 0x675b2a1 thru 0x676b1e6 (0xff46 bytes) 2024-04-23T23:50:28.510 INFO:teuthology.orchestra.run.smithi092.stdout:3315 trunc from 0x722b0db to 0xa7dc73b 2024-04-23T23:50:28.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:28.516 INFO:teuthology.orchestra.run.smithi092.stdout:3316 writesame 0x6dbf260 thru 0x6dc6ccd data_size 0x1(0x7a6e bytes) 2024-04-23T23:50:28.538 INFO:teuthology.orchestra.run.smithi092.stdout:3317 read 0x3c456a4 thru 0x3c4f553 (0x9eb0 bytes) 2024-04-23T23:50:28.538 INFO:teuthology.orchestra.run.smithi092.stdout:3319 clone 22 order 19 su 16384 sc 2 2024-04-23T23:50:28.557 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:28.571+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:28.623 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:28.623 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:28.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:28.657 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:28.671+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:28.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:28.759 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:28.772+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:28.776 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:28.776 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:28.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:28.847 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:28.859 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:28.872+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:28.864 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:28.882 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:28.883 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:28.885 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:566 3.4 e28/22 3:23c10081:::rbd_data.1179d7437ec6.0000000000000027:head v 28'3, mlcod=28'1)", 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.990095837999998, 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001175798, 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915527+0000", 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915528+0000", 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0580000000000001e-06 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915539+0000", 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638999999999999e-05 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915554+0000", 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5141e-05 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915565+0000", 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0789e-05 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915620+0000", 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5189999999999998e-05 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.915642+0000", 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2262000000000001e-05 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916672+0000", 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010296109999999999 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:56.916703+0000", 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1108000000000001e-05 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1147 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'4, mlcod=28'2)", 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.974011706999999, 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022957910000000002, 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931611+0000", 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931628+0000", 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6813999999999999e-05 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931636+0000", 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931644+0000", 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3359999999999997e-06 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931703+0000", 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9598000000000001e-05 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.931724+0000", 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0735000000000001e-05 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933865+0000", 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002141165 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:57.933907+0000", 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1768999999999999e-05 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1526 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'16, mlcod=29'14)", 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.51994191, 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021410930000000002, 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385681+0000", 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385682+0000", 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385693+0000", 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0998999999999999e-05 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385710+0000", 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7280000000000001e-05 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385719+0000", 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7940000000000008e-06 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385785+0000", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6109e-05 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.385807+0000", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-05 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387778+0000", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019710769999999999 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.387822+0000", 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3665000000000003e-05 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:1728 3.4 e29/22 3:26945585:::rbd_data.1179984961fa.0000000000000006:head v 29'20, mlcod=29'18)", 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.290063594999999, 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022012160000000002, 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615559+0000", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615560+0000", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.136e-06 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615580+0000", 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9409e-05 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615597+0000", 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.772e-05 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615606+0000", 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6780000000000006e-06 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615677+0000", 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0714000000000001e-05 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.615701+0000", 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4076000000000001e-05 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617717+0000", 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020162729999999999 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:49:58.617760+0000", 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3210000000000001e-05 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.446060578000001, 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3876 3.9 e33/22 3:93f02760:::rbd_data.1179c62058ea.0000000000000008:head v 33'13, mlcod=33'10)", 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.437909507000001, 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014124350000000001, 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467713+0000", 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467714+0000", 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1100000000000004e-07 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467723+0000", 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0820000000000005e-06 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467738+0000", 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4598e-05 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467746+0000", 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0620000000000001e-06 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467806+0000", 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0591000000000003e-05 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.467827+0000", 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0282e-05 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469094+0000", 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001267481 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.469126+0000", 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1427999999999998e-05 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4487 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'62, mlcod=32'60)", 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.241584888999999, 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020224290000000001, 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664038+0000", 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0599999999999999e-07 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664042+0000", 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.501e-06 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664051+0000", 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6609999999999995e-06 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664057+0000", 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6420000000000004e-06 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664109+0000", 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.664129+0000", 2024-04-23T23:50:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0041999999999999e-05 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666021+0000", 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00189174 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.666060+0000", 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9149000000000001e-05 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:4629 3.0 e33/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 33'4, mlcod=31'2)", 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.226147561000001, 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002138633, 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679475+0000", 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679476+0000", 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1999999999999998e-07 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679583+0000", 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106983 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679599+0000", 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962999999999999e-05 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679608+0000", 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6659999999999997e-06 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679662+0000", 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4385000000000002e-05 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.679685+0000", 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2667000000000002e-05 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681580+0000", 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018954270000000001 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.681614+0000", 2024-04-23T23:50:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3722e-05 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5343 3.4 e33/22 3:238ba95a:::rbd_data.1179c4f11774.0000000000000010:head v 33'64, mlcod=33'62)", 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.982148927000001, 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020278729999999999, 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923474+0000", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923480+0000", 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923490+0000", 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3919999999999992e-06 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923497+0000", 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923545+0000", 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8647999999999999e-05 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.923560+0000", 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5065000000000001e-05 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925471+0000", 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019111340000000001 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:02.925502+0000", 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0321e-05 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.698401617999998, 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.4 epoch 36/22 rep_tid 124 entries 36'70 (0'0) error 3:25e2e575:::rbd_data.117967c411b0.000000000000002e:head by client.4473.0:6553 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'68)", 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.886557100000001, 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027723230000000001, 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019066+0000", 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019067+0000", 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9599999999999998e-07 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019079+0000", 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2716e-05 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019107+0000", 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8232999999999999e-05 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019125+0000", 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717e-05 2024-04-23T23:50:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.019219+0000", 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3455999999999994e-05 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.021838+0000", 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026193050000000002 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 36/22 rep_tid 127 entries 36'23 (0'0) error 3:05311e05:::rbd_data.117967c411b0.0000000000000034:head by client.4473.0:6561 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'21)", 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.883167803999999, 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002186155, 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022455+0000", 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022456+0000", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7899999999999998e-07 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022464+0000", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022484+0000", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0369000000000001e-05 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022495+0000", 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1205e-05 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.022559+0000", 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3465000000000003e-05 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.024641+0000", 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020822810000000001 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6580 3.0 e36/22 3:053657a9:::rbd_data.117967c411b0.000000000000003a:head v 36'26, mlcod=36'24)", 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.874305277000001, 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011394210000000001, 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031317+0000", 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031318+0000", 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-07 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031324+0000", 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031338+0000", 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4036000000000001e-05 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031346+0000", 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2450000000000001e-06 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031392+0000", 2024-04-23T23:50:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6214000000000002e-05 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.031410+0000", 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7859e-05 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032426+0000", 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010161530000000001 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.032457+0000", 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1151e-05 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.856532312999999, 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:24539 3.9 e42/22 3:908cbee1:::rbd_data.1179cfb6b7da.0000000000000031:head v 42'202, mlcod=42'200)", 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:14.715977+0000", 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.189645977, 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013857749999999999, 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715977+0000", 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715977+0000", 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715978+0000", 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1289999999999999e-06 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.715989+0000", 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1306e-05 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716006+0000", 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7110999999999999e-05 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716015+0000", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7590000000000003e-06 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716447+0000", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043230100000000001 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.716468+0000", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0873000000000002e-05 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.717319+0000", 2024-04-23T23:50:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085117600000000004 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.717363+0000", 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3120000000000001e-05 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:24639 3.9 e42/22 3:908cbee1:::rbd_data.1179cfb6b7da.0000000000000031:head v 42'204, mlcod=42'202)", 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:14.826550+0000", 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.079072915999999, 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011463809999999999, 2024-04-23T23:50:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826550+0000", 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826550+0000", 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826552+0000", 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2170000000000001e-06 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826568+0000", 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5855999999999999e-05 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826600+0000", 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1742999999999997e-05 2024-04-23T23:50:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826617+0000", 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7351999999999999e-05 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826761+0000", 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014441599999999999 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.826822+0000", 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0732000000000001e-05 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.827657+0000", 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083490400000000003 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:14.827696+0000", 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9161e-05 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:24916 3.9 e43/22 3:94620a61:::rbd_data.1179cfb6b7da.000000000000002f:head v 43'209, mlcod=42'206)", 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.850582077, 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011846070000000001, 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055041+0000", 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6899999999999997e-07 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055048+0000", 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6309999999999997e-06 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055057+0000", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4609999999999997e-06 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055064+0000", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1500000000000002e-06 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055289+0000", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000224868 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.055312+0000", 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2949e-05 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.056192+0000", 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087976099999999995 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:16.056225+0000", 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3217999999999999e-05 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:31114 3.4 e47/22 3:25439e5d:::rbd_data.11797b718ff4.000000000000000a:head v 47'82, mlcod=44'80)", 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:21.583198+0000", 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3224245889999997, 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001416004, 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583198+0000", 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583198+0000", 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583199+0000", 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.251e-06 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583208+0000", 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9190000000000005e-06 2024-04-23T23:50:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583229+0000", 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0611e-05 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583239+0000", 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6560000000000004e-06 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583297+0000", 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8721000000000002e-05 2024-04-23T23:50:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.583319+0000", 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1569e-05 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.584572+0000", 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001253299 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:21.584614+0000", 2024-04-23T23:50:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1977999999999997e-05 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:40806 3.9 e50/22 3:91237d06:::rbd_data.11795f20a36d.000000000000019b:head v 50'250, mlcod=50'246)", 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.671853+0000", 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2337700790000001, 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001622163, 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671853+0000", 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671853+0000", 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671853+0000", 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8999999999999996e-07 2024-04-23T23:50:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671860+0000", 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2180000000000002e-06 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671871+0000", 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0151e-05 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671877+0000", 2024-04-23T23:50:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.567e-06 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671918+0000", 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0571000000000002e-05 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.671935+0000", 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7691999999999999e-05 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.673425+0000", 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001489181 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.673475+0000", 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0192999999999999e-05 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:51729 3.9 e51/22 3:920f0cce:::rbd_data.1179cce21f00.0000000000000007:head v 51'257, mlcod=51'255)", 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:26.243201+0000", 2024-04-23T23:50:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6624212959999998, 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011380369999999999, 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243201+0000", 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243201+0000", 2024-04-23T23:50:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243204+0000", 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999998e-06 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243242+0000", 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7679000000000003e-05 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243275+0000", 2024-04-23T23:50:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3158999999999999e-05 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243291+0000", 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5809999999999999e-05 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243413+0000", 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012184 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.243453+0000", 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9622000000000001e-05 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.244297+0000", 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084478099999999996 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:26.244339+0000", 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2166000000000003e-05 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:28.978 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:28.979 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:28.979 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:28.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:28.980 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:28.981 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:28.981 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:28.981 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:28.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:28.983 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:28.983 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:28.993+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:29.080 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:29.093+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:29.086 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:29.086 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:29.086 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:29.086 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:29.088 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:50:29.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:29.161 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone21 2024-04-23T23:50:29.183 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:29.196+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:29.263 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:29.263 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:29.263 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:29.264 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:29.264 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:29.273 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:29.281 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:29.295+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:29.382 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:29.395+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:29.456 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:29.456 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:29.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:29.456 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0380333640000003, 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.457 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:50:29.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.038023484, 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:50:29.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:50:29.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0378131369999997, 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:50:29.465 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0359817979999999, 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.470 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:50:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:29.474 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0339121670000004, 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:50:29.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:50:29.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5218219309999999, 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:50:29.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:50:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.483 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5217962749999998, 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:50:29.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:50:29.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5217287849999996, 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5216837239999998, 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:50:29.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:29.496 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5216472239999996, 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5216302649999998, 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5215515809999998, 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:50:29.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:50:29.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5214845480000001, 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:50:29.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5213509470000002, 2024-04-23T23:50:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:50:29.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5209322670000001, 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:50:29.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:50:29.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:50:29.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5207816249999997, 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.524 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5206562430000004, 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5204407260000004, 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.520203392, 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:50:29.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.519963497, 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.541 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:50:29.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.545 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:29.547 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:29.560+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:29.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:29.648 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:29.661+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:29.649 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:29.649 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:29.658 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:29.681 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-04-23T23:50:29.682 INFO:teuthology.orchestra.run.smithi092.stdout:3320 punch from 0x7bcc7d4 to 0x7bda342, (0xdb6e bytes) 2024-04-23T23:50:29.685 INFO:teuthology.orchestra.run.smithi092.stdout:3321 read 0x3fa9e0e thru 0x3fae033 (0x4226 bytes) 2024-04-23T23:50:29.692 INFO:teuthology.orchestra.run.smithi092.stdout:3322 write 0xbfcdd7a thru 0xbfd6f30 (0x91b7 bytes) 2024-04-23T23:50:29.695 INFO:teuthology.orchestra.run.smithi092.stdout:3327 punch from 0x3a1d32b to 0x3a21432, (0x4107 bytes) 2024-04-23T23:50:29.695 INFO:teuthology.orchestra.run.smithi092.stdout:3329 write 0xaefb7c7 thru 0xaf04617 (0x8e51 bytes) 2024-04-23T23:50:29.701 INFO:teuthology.orchestra.run.smithi092.stdout:3330 write 0x824193e thru 0x824bef0 (0xa5b3 bytes) 2024-04-23T23:50:29.704 INFO:teuthology.orchestra.run.smithi092.stdout:3331 read 0x1c72357 thru 0x1c76b84 (0x482e bytes) 2024-04-23T23:50:29.704 INFO:teuthology.orchestra.run.smithi092.stdout:3332 write 0x84b3d60 thru 0x84b4190 (0x431 bytes) 2024-04-23T23:50:29.710 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:29.710 INFO:teuthology.orchestra.run.smithi092.stdout:3334 write 0x2fe41c0 thru 0x2fe766a (0x34ab bytes) 2024-04-23T23:50:29.710 INFO:teuthology.orchestra.run.smithi092.stdout:3336 write 0x6619df8 thru 0x6626ad6 (0xccdf bytes) 2024-04-23T23:50:29.721 INFO:teuthology.orchestra.run.smithi092.stdout:3337 writesame 0xe128b6f thru 0xe12c074 data_size 0xb(0x3506 bytes) 2024-04-23T23:50:29.722 INFO:teuthology.orchestra.run.smithi092.stdout:3338 read 0x3f9bb79 thru 0x3f9d737 (0x1bbf bytes) 2024-04-23T23:50:29.722 INFO:teuthology.orchestra.run.smithi092.stdout:3340 read 0x6e5b970 thru 0x6e62020 (0x66b1 bytes) 2024-04-23T23:50:29.722 INFO:teuthology.orchestra.run.smithi092.stdout:3341 writesame 0xd4310b thru 0xd50053 data_size 0x1(0xcf49 bytes) 2024-04-23T23:50:29.749 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:29.763+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:29.753 INFO:teuthology.orchestra.run.smithi092.stdout:3343 punch from 0x28a31cf to 0x28ae469, (0xb29a bytes) 2024-04-23T23:50:29.753 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:29.754 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:29.754 INFO:teuthology.orchestra.run.smithi092.stdout:3347 read 0xd013bec thru 0xd023470 (0xf885 bytes) 2024-04-23T23:50:29.754 INFO:teuthology.orchestra.run.smithi092.stdout:3348 writesame 0xba6067e thru 0xba6ca66 data_size 0x1(0xc3e9 bytes) 2024-04-23T23:50:29.789 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:29.789 INFO:teuthology.orchestra.run.smithi092.stdout:3350 read 0xa15e6f8 thru 0xa16b39a (0xcca3 bytes) 2024-04-23T23:50:29.790 INFO:teuthology.orchestra.run.smithi092.stdout:3351 write 0x854d955 thru 0x855aad3 (0xd17f bytes) 2024-04-23T23:50:29.793 INFO:teuthology.orchestra.run.smithi092.stdout:3352 punch from 0x3b72926 to 0x3b80a12, (0xe0ec bytes) 2024-04-23T23:50:29.793 INFO:teuthology.orchestra.run.smithi092.stdout:3353 writesame 0x2ba18d1 thru 0x2ba6c94 data_size 0x6(0x53c4 bytes) 2024-04-23T23:50:29.798 INFO:teuthology.orchestra.run.smithi092.stdout:3355 write 0xa08512e thru 0xa08a020 (0x4ef3 bytes) 2024-04-23T23:50:29.801 INFO:teuthology.orchestra.run.smithi092.stdout:3356 clone 23 order 19 su 131072 sc 9 2024-04-23T23:50:29.806 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:29.806 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:29.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:29.850 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:29.864+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:29.919 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:29.919 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.66214518700000002, 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.920 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:29.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.66212430700000002, 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65626652100000005, 2024-04-23T23:50:29.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:50:29.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65625227399999997, 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.935 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:50:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:50:29.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65421834099999998, 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:50:29.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:50:29.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65420639899999999, 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.944 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:50:29.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.652229265, 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:50:29.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65220118100000002, 2024-04-23T23:50:29.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:29.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:50:29.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:50:29.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:50:29.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65218864600000004, 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.958 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:50:29.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:50:29.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.650247558, 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.966 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65023621200000004, 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:50:29.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:50:29.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.971 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65022535800000003, 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:50:29.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:50:29.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:50:29.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:50:29.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.978 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.978 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:29.978 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.65021406699999995, 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:50:29.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:29.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:50:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:50:29.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:50:29.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:50:29.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:50:29.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:50:29.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:50:29.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:50:29.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:50:29.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:50:29.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:50:29.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:50:29.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.64817529399999996, 2024-04-23T23:50:29.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:50:29.989 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:29.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:29.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:29.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:29.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:29.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:29.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:29.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:50:29.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:29.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:50:29.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:50:29.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:29.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:50:29.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:50:29.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:29.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:50:29.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:50:29.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:29.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:50:29.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:50:29.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:29.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:50:29.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:50:29.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:29.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:29.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:29.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:29.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:29.999 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:29.999 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.64815931100000002, 2024-04-23T23:50:29.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:50:29.999 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:29.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.000 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.000 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.000 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.000 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:50:30.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:30.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:50:30.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:50:30.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:50:30.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:50:30.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:50:30.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:50:30.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:50:30.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:50:30.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:50:30.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:50:30.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:50:30.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:50:30.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.009 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:30.009 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:30.009 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.64813869199999996, 2024-04-23T23:50:30.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:50:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:50:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:50:30.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.648118694, 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:30.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:50:30.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.64810041399999996, 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:50:30.019 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:50:30.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.64709821599999995, 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:30.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:50:30.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:50:30.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.64708671699999998, 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:50:30.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.034 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:30.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:30.036 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:30.049+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:30.137 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:30.150+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:30.146 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:30.146 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:30.146 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:30.146 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:30.150 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone22 2024-04-23T23:50:30.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:30.238 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:30.251+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:30.285 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:30.285 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:30.285 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:30.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:30.378 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:30.391+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:30.381 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-04-23T23:50:30.440 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:30.454+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9452767680000003, 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:50:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:50:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9452437199999997, 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:50:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:50:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9452212209999997, 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:50:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9433513040000001, 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:50:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9433228280000003, 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:50:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:50:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9433003840000005, 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:50:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9413446560000001, 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:50:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9413255290000002, 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9412993810000003, 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:50:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:50:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9412786889999998, 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:50:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9412569629999998, 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9393600729999996, 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:50:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:50:30.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:50:30.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9393356260000001, 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:30.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9393185319999997, 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:30.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:50:30.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:50:30.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.937341151, 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:50:30.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:50:30.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9373222730000004, 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:50:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:50:30.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:50:30.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:50:30.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9353412329999999, 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:50:30.531 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:30.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:30.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:50:30.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:50:30.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:50:30.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:50:30.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:50:30.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:50:30.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:50:30.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:50:30.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.540 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.540 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:30.540 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9353014310000001, 2024-04-23T23:50:30.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:50:30.540 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:50:30.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:50:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:50:30.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:50:30.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:50:30.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:50:30.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:50:30.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:50:30.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:50:30.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:50:30.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.550 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:30.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9352794070000003, 2024-04-23T23:50:30.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:50:30.550 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.551 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.551 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:50:30.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:30.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:30.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:50:30.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:50:30.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:50:30.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:50:30.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:50:30.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9352578659999997, 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:30.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:30.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:50:30.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:50:30.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:50:30.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:50:30.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:30.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:30.564 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:30.566 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:30.566 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:30.578+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:30.621 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:30.635+0000 7efdcc218640 -1 osd.3 38 log_to_monitors true 2024-04-23T23:50:30.624 INFO:teuthology.orchestra.run.smithi092.stdout:3357 trunc from 0xe12c075 to 0xe6054ae 2024-04-23T23:50:30.646 INFO:teuthology.orchestra.run.smithi092.stdout:3358 trunc from 0xe6054ae to 0x232c4b3 2024-04-23T23:50:30.666 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:30.679+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:30.682 INFO:teuthology.orchestra.run.smithi092.stdout:3359 write 0xcef24ac thru 0xcefb5e1 (0x9136 bytes) 2024-04-23T23:50:30.685 INFO:teuthology.orchestra.run.smithi092.stdout:3361 read 0xa075bf3 thru 0xa083dc8 (0xe1d6 bytes) 2024-04-23T23:50:30.686 INFO:teuthology.orchestra.run.smithi092.stdout:3364 writesame 0x897fa3c thru 0x898c1e7 data_size 0x1a(0xc7ac bytes) 2024-04-23T23:50:30.693 INFO:teuthology.orchestra.run.smithi092.stdout:3365 read 0x8429cf5 thru 0x842ae71 (0x117d bytes) 2024-04-23T23:50:30.693 INFO:teuthology.orchestra.run.smithi092.stdout:3366 punch from 0x9f60bb0 to 0x9f6c1b3, (0xb603 bytes) 2024-04-23T23:50:30.693 INFO:teuthology.orchestra.run.smithi092.stdout:3367 write 0x2f75d9f thru 0x2f7c2e8 (0x654a bytes) 2024-04-23T23:50:30.695 INFO:teuthology.orchestra.run.smithi092.stdout:3368 writesame 0x65747f4 thru 0x657d141 data_size 0x19(0x894e bytes) 2024-04-23T23:50:30.711 INFO:teuthology.orchestra.run.smithi092.stdout:3370 trunc from 0xcefb5e2 to 0x20b382d 2024-04-23T23:50:30.718 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:30.718 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:30.718 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:30.718 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:30.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:30.730 INFO:teuthology.orchestra.run.smithi092.stdout:3371 punch from 0x1ab7fad to 0x1ac3bec, (0xbc3f bytes) 2024-04-23T23:50:30.743 INFO:teuthology.orchestra.run.smithi092.stdout:3372 write 0x8b844f4 thru 0x8b9018b (0xbc98 bytes) 2024-04-23T23:50:30.746 INFO:teuthology.orchestra.run.smithi092.stdout:3374 read 0x16fea89 thru 0x1704eb1 (0x6429 bytes) 2024-04-23T23:50:30.752 INFO:teuthology.orchestra.run.smithi092.stdout:3376 trunc from 0x8b9018c to 0x35f6067 2024-04-23T23:50:30.765 INFO:teuthology.orchestra.run.smithi092.stdout:3377 trunc from 0x35f6067 to 0x697d9bd 2024-04-23T23:50:30.769 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:30.782+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:30.770 INFO:teuthology.orchestra.run.smithi092.stdout:3378 write 0x45376ce thru 0x453e061 (0x6994 bytes) 2024-04-23T23:50:30.774 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:30.774 INFO:teuthology.orchestra.run.smithi092.stdout:3381 read 0x371999a thru 0x371a993 (0xffa bytes) 2024-04-23T23:50:30.774 INFO:teuthology.orchestra.run.smithi092.stdout:3384 read 0x60162b9 thru 0x601ea5c (0x87a4 bytes) 2024-04-23T23:50:30.774 INFO:teuthology.orchestra.run.smithi092.stdout:3385 punch from 0x52fe326 to 0x5305458, (0x7132 bytes) 2024-04-23T23:50:30.779 INFO:teuthology.orchestra.run.smithi092.stdout:3386 trunc from 0x697d9bd to 0x2338488 2024-04-23T23:50:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:3388 read 0x2a56f6 thru 0x2b21d9 (0xcae4 bytes) 2024-04-23T23:50:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:3389 punch from 0x1ed470 to 0x1f5c64, (0x87f4 bytes) 2024-04-23T23:50:30.796 INFO:teuthology.orchestra.run.smithi092.stdout:3390 write 0x3c96ca5 thru 0x3ca43bf (0xd71b bytes) 2024-04-23T23:50:30.799 INFO:teuthology.orchestra.run.smithi092.stdout:3391 read 0x277bef8 thru 0x27886ad (0xc7b6 bytes) 2024-04-23T23:50:30.799 INFO:teuthology.orchestra.run.smithi092.stdout:3393 read 0x11714ca thru 0x117f043 (0xdb7a bytes) 2024-04-23T23:50:30.815 INFO:teuthology.orchestra.run.smithi092.stdout:3394 write 0xc97d310 thru 0xc9888fe (0xb5ef bytes) 2024-04-23T23:50:30.830 INFO:teuthology.orchestra.run.smithi092.stdout:3396 trunc from 0xc9888ff to 0x2d13bef 2024-04-23T23:50:30.851 INFO:teuthology.orchestra.run.smithi092.stdout:3397 writesame 0x83ac015 thru 0x83bb105 data_size 0x1(0xf0f1 bytes) 2024-04-23T23:50:30.856 INFO:teuthology.orchestra.run.smithi092.stdout:3398 trunc from 0x83bb106 to 0xcee3f3d 2024-04-23T23:50:30.861 INFO:teuthology.orchestra.run.smithi092.stdout:3399 read 0x5b2827c thru 0x5b33fdf (0xbd64 bytes) 2024-04-23T23:50:30.861 INFO:teuthology.orchestra.run.smithi092.stdout:3400 write 0x9a2ecff thru 0x9a3dded (0xf0ef bytes) 2024-04-23T23:50:30.865 INFO:teuthology.orchestra.run.smithi092.stdout:3401 read 0x908f17d thru 0x90965ee (0x7472 bytes) 2024-04-23T23:50:30.868 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:30.881+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:30.876 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:30.876 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:30.876 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:30.879 INFO:teuthology.orchestra.run.smithi092.stdout:3404 trunc from 0xcee3f3d to 0xd845e6 2024-04-23T23:50:30.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:30.898 INFO:teuthology.orchestra.run.smithi092.stdout:3405 write 0x20e15cf thru 0x20e5910 (0x4342 bytes) 2024-04-23T23:50:30.901 INFO:teuthology.orchestra.run.smithi092.stdout:3406 punch from 0x1e91f00 to 0x1e94dea, (0x2eea bytes) 2024-04-23T23:50:30.901 INFO:teuthology.orchestra.run.smithi092.stdout:3407 punch from 0x112e287 to 0x113135c, (0x30d5 bytes) 2024-04-23T23:50:30.901 INFO:teuthology.orchestra.run.smithi092.stdout:3408 read 0x29c8ff thru 0x29e67f (0x1d81 bytes) 2024-04-23T23:50:30.914 INFO:teuthology.orchestra.run.smithi092.stdout:3409 writesame 0x89b780a thru 0x89c47aa data_size 0x1(0xcfa1 bytes) 2024-04-23T23:50:30.922 INFO:teuthology.orchestra.run.smithi092.stdout:3411 write 0xbd23c5d thru 0xbd253e3 (0x1787 bytes) 2024-04-23T23:50:30.924 INFO:teuthology.orchestra.run.smithi092.stdout:3412 write 0x11e484d thru 0x11e9bfa (0x53ae bytes) 2024-04-23T23:50:30.927 INFO:teuthology.orchestra.run.smithi092.stdout:3414 write 0xa298d38 thru 0xa2a104a (0x8313 bytes) 2024-04-23T23:50:30.930 INFO:teuthology.orchestra.run.smithi092.stdout:3415 read 0x3642e54 thru 0x36528a4 (0xfa51 bytes) 2024-04-23T23:50:30.930 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:30.930 INFO:teuthology.orchestra.run.smithi092.stdout:3417 write 0x2d9556c thru 0x2d9bb2f (0x65c4 bytes) 2024-04-23T23:50:30.938 INFO:teuthology.orchestra.run.smithi092.stdout:3418 trunc from 0xbd253e4 to 0x7a4fb26 2024-04-23T23:50:30.945 INFO:teuthology.orchestra.run.smithi092.stdout:3419 punch from 0x6bf8334 to 0x6c0781a, (0xf4e6 bytes) 2024-04-23T23:50:30.953 INFO:teuthology.orchestra.run.smithi092.stdout:3420 trunc from 0x7a4fb26 to 0xe84b6d9 2024-04-23T23:50:30.959 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:30.959 INFO:teuthology.orchestra.run.smithi092.stdout:3422 writesame 0xe5410e7 thru 0xe54498e data_size 0x4a(0x38a8 bytes) 2024-04-23T23:50:30.961 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:30.962 INFO:teuthology.orchestra.run.smithi092.stdout:3426 write 0x1afe7e4 thru 0x1afea19 (0x236 bytes) 2024-04-23T23:50:30.969 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:30.982+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:30.969 INFO:teuthology.orchestra.run.smithi092.stdout:3427 write 0xece415f thru 0xece9cb2 (0x5b54 bytes) 2024-04-23T23:50:30.972 INFO:teuthology.orchestra.run.smithi092.stdout:3428 read 0xd62648 thru 0xd66aac (0x4465 bytes) 2024-04-23T23:50:30.972 INFO:teuthology.orchestra.run.smithi092.stdout:3430 read 0x9f76c5b thru 0x9f8078b (0x9b31 bytes) 2024-04-23T23:50:30.972 INFO:teuthology.orchestra.run.smithi092.stdout:3431 clone 24 order 18 su 4096 sc 6 2024-04-23T23:50:30.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:31.033 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.845742577999999, 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.497005909999999, 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.496989457000002, 2024-04-23T23:50:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:31.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.496962715999999, 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.496943921, 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:31.057 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.496926337000001, 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:31.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:31.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.485230832999999, 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.062 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:31.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.066 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.485055146000001, 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.067 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:31.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:31.071 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.484903627000001, 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:31.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.484621001000001, 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.080 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.484475136, 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:31.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:31.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:31.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.483525451999999, 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.483502558000001, 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.467487182999999, 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:31.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.467477549000002, 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:31.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:31.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:31.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.452445187999999, 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:31.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.452423864, 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:31.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:31.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.112 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.452401039000002, 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.452378360000001, 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:31.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:31.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:31.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.450076636999999, 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:31.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:31.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:31.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:31.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.128 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:31.128 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:31.128 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:31.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:50:31.129 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:31.130 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:31.130 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:31.127+0000 7efdbfa8c640 -1 osd.3 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T23:50:31.130 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:31.131 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:31.131 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:31.143+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:31.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:31.146 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:31.146 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-04-23T23:50:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone23 from 0xece9cb3 (overlap 0xd845e6) to 0x976421 2024-04-23T23:50:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:31.166 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-04-23T23:50:31.230 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:31.243+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:31.232 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:31.266 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:31.270 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-23T23:50:31.270 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-23T23:50:31.286 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:31.286 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:31.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:31.345 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:31.359+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:31.432 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:31.445+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:31.455 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:31.456 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:31.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:31.533 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:31.547+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:31.553 INFO:teuthology.orchestra.run.smithi092.stdout:3432 write 0x392cb6 thru 0x39a7be (0x7b09 bytes) 2024-04-23T23:50:31.581 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:31.581 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:31.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:31.600 INFO:teuthology.orchestra.run.smithi092.stdout:3433 write 0x757b4e9 thru 0x75840de (0x8bf6 bytes) 2024-04-23T23:50:31.609 INFO:teuthology.orchestra.run.smithi092.stdout:3435 read 0x54c8fce thru 0x54d32c1 (0xa2f4 bytes) 2024-04-23T23:50:31.610 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:31.610 INFO:teuthology.orchestra.run.smithi092.stdout:3437 trunc from 0xece9cb3 to 0xe38dd4f 2024-04-23T23:50:31.624 INFO:teuthology.orchestra.run.smithi092.stdout:3438 punch from 0xd08b806 to 0xd094715, (0x8f0f bytes) 2024-04-23T23:50:31.625 INFO:teuthology.orchestra.run.smithi092.stdout:3439 write 0x5fe4abc thru 0x5fed30e (0x8853 bytes) 2024-04-23T23:50:31.631 INFO:teuthology.orchestra.run.smithi092.stdout:3441 write 0xaab7d1b thru 0xaabf6ca (0x79b0 bytes) 2024-04-23T23:50:31.634 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:31.647+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:31.638 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:31.639 INFO:teuthology.orchestra.run.smithi092.stdout:3444 write 0x5e8b7c5 thru 0x5e8daff (0x233b bytes) 2024-04-23T23:50:31.640 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:31.640 INFO:teuthology.orchestra.run.smithi092.stdout:3446 read 0x72e2075 thru 0x72eacd1 (0x8c5d bytes) 2024-04-23T23:50:31.640 INFO:teuthology.orchestra.run.smithi092.stdout:3447 write 0x5d93277 thru 0x5d9e31b (0xb0a5 bytes) 2024-04-23T23:50:31.646 INFO:teuthology.orchestra.run.smithi092.stdout:3448 writesame 0xa4c4ba7 thru 0xa4ca2a4 data_size 0x22(0x56fe bytes) 2024-04-23T23:50:31.657 INFO:teuthology.orchestra.run.smithi092.stdout:3450 write 0x313972b thru 0x3143eab (0xa781 bytes) 2024-04-23T23:50:31.670 INFO:teuthology.orchestra.run.smithi092.stdout:3452 writesame 0xb3b0380 thru 0xb3b7ee1 data_size 0x11(0x7b62 bytes) 2024-04-23T23:50:31.677 INFO:teuthology.orchestra.run.smithi092.stdout:3453 writesame 0xb4314ab thru 0xb43cc91 data_size 0x3(0xb7e7 bytes) 2024-04-23T23:50:31.695 INFO:teuthology.orchestra.run.smithi092.stdout:3454 trunc from 0xe38dd4f to 0xa5aecb0 2024-04-23T23:50:31.714 INFO:teuthology.orchestra.run.smithi092.stdout:3455 writesame 0x817f9f8 thru 0x818e658 data_size 0x17(0xec61 bytes) 2024-04-23T23:50:31.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:31.724 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:31.724 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:31.728 INFO:teuthology.orchestra.run.smithi092.stdout:3456 write 0xb7959f9 thru 0xb7a29fc (0xd004 bytes) 2024-04-23T23:50:31.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:31.735 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:31.747+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:31.738 INFO:teuthology.orchestra.run.smithi092.stdout:3457 punch from 0xa2cd534 to 0xa2da777, (0xd243 bytes) 2024-04-23T23:50:31.743 INFO:teuthology.orchestra.run.smithi092.stdout:3458 write 0xc430265 thru 0xc431d50 (0x1aec bytes) 2024-04-23T23:50:31.751 INFO:teuthology.orchestra.run.smithi092.stdout:3459 write 0x4c6815a thru 0x4c7792b (0xf7d2 bytes) 2024-04-23T23:50:31.761 INFO:teuthology.orchestra.run.smithi092.stdout:3460 writesame 0x2269696 thru 0x2275198 data_size 0x19(0xbb03 bytes) 2024-04-23T23:50:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:3463 write 0x69180e2 thru 0x6919f78 (0x1e97 bytes) 2024-04-23T23:50:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:3465 trunc from 0xc431d51 to 0xcf6d2c6 2024-04-23T23:50:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:3466 read 0x8c3efa9 thru 0x8c49a3d (0xaa95 bytes) 2024-04-23T23:50:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:3468 trunc from 0xcf6d2c6 to 0x53bbfe0 2024-04-23T23:50:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:3470 punch from 0x450ed9d to 0x451cc83, (0xdee6 bytes) 2024-04-23T23:50:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:3472 trunc from 0x53bbfe0 to 0xc3137e4 2024-04-23T23:50:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:3473 punch from 0x4bf98af to 0x4bff478, (0x5bc9 bytes) 2024-04-23T23:50:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:3475 punch from 0xb6056b to 0xb68a47, (0x84dc bytes) 2024-04-23T23:50:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:3476 trunc from 0xc3137e4 to 0x65228bd 2024-04-23T23:50:31.835 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:31.849+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:31.845 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:31.845 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:3477 writesame 0x96b3e28 thru 0x96b6925 data_size 0x1(0x2afe bytes) 2024-04-23T23:50:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:3478 write 0xe866d95 thru 0xe86e472 (0x76de bytes) 2024-04-23T23:50:31.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:3480 write 0x6017e14 thru 0x6025504 (0xd6f1 bytes) 2024-04-23T23:50:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:3481 writesame 0x943078a thru 0x9437e37 data_size 0xb(0x76ae bytes) 2024-04-23T23:50:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:3482 punch from 0xa771f8d to 0xa777491, (0x5504 bytes) 2024-04-23T23:50:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:3483 punch from 0xe2cc3f6 to 0xe2d2eda, (0x6ae4 bytes) 2024-04-23T23:50:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:3486 read 0x8d1d1bb thru 0x8d29da3 (0xcbe9 bytes) 2024-04-23T23:50:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:3487 writesame 0x1c37576 thru 0x1c42df3 data_size 0x5(0xb87e bytes) 2024-04-23T23:50:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:3488 read 0x6ff779e thru 0x6ff9845 (0x20a8 bytes) 2024-04-23T23:50:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:3490 clone 25 order 20 su 8192 sc 7 2024-04-23T23:50:31.936 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:31.950+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:31.977 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:31.977 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:31.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:32.037 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:32.050+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:50:32.136 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.136 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.137 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:32.151+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:32.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:32.238 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:32.252+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:32.287 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.287 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:32.338 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:32.352+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:32.439 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:32.453+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:32.440 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.440 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:32.540 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:32.553+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:32.553 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.553 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:32.646 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:32.659+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:32.704 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.704 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:32.754 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:32.767+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:32.843 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:32.856+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:32.855 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.855 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:32.944 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:32.957+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:32.970 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:32.970 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:32.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:33.044 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:33.057+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:33.083 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:33.084 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:33.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:33.176 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:33.189+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:33.235 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:33.235 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:33.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:33.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:33.285 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:33.298+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:33.347 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:33.360+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:33.388 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:33.388 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:33.391 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:33.391 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:33.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:33.408 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:33.409 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:33.425 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:33.425 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:33.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:33.442 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-04-23T23:50:33.459 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:33.476 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:33.480 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:33.493+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:33.492 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:33.509 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:33.540 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:33.540 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:33.549 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:33.549 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:33.561+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:33.649 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:33.663+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:33.693 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:33.693 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:33.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:33.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:33.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.693 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1873486440000001, 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.697 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1873155959999995, 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.698 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1872930969999995, 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.702 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.706 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1854231800000008, 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.710 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1853947040000001, 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.711 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:50:33.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:50:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1853722599999994, 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.715 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:50:33.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1834165320000007, 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.719 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:33.723 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1833974050000009, 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:50:33.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1833712569999992, 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:50:33.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1833505649999996, 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:50:33.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:33.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1833288389999996, 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:50:33.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:50:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1814319490000003, 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:33.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:50:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1814075020000008, 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:50:33.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1813904080000004, 2024-04-23T23:50:33.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:50:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1794130270000007, 2024-04-23T23:50:33.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.756 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1793941490000002, 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:33.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1774131089999997, 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:50:33.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:50:33.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1773733069999999, 2024-04-23T23:50:33.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:33.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1773512830000001, 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:50:33.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:50:33.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1773297419999995, 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:33.774 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:33.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:33.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:33.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:33.778 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:33.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:33.780 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:33.793+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:33.881 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:33.894+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:33.923 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:33.924 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:33.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:34.016 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:34.029+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:34.076 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:34.076 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:34.076 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:34.076 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:34.076 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:34.082 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:34.095+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:34.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:34.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:50:34.145 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone24 from 0xe86e473 (overlap 0x53bbfe0) to 0x0 2024-04-23T23:50:34.183 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:34.196+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:34.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.083157036000003, 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:34.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:34.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:34.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:34.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:34.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.734420367999999, 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:34.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.734403915000001, 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:34.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:34.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.734377173999999, 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:34.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.734358379, 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:34.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:34.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.298 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.734340795000001, 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:34.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:34.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.722645290999999, 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.303 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:34.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.722469604, 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.722318085000001, 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.722035459000001, 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.721889594, 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:34.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:34.324 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.720939909999998, 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:34.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:34.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.720917016000001, 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:34.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:34.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.704901640999999, 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.704892007000002, 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:34.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.689859645999999, 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:34.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.689838322, 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:34.350 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.689815497000001, 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.689792818000001, 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:34.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:34.358 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.687491094999999, 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:34.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:34.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:34.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:34.364 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:34.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:34.366 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:34.379+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:34.466 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:34.480+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:34.522 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:34.522 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:34.522 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:34.522 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:34.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:34.577 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:34.591+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:34.612 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-04-23T23:50:34.635 INFO:teuthology.orchestra.run.smithi092.stdout:3491 read 0x54248de thru 0x542cd69 (0x848c bytes) 2024-04-23T23:50:34.635 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:34.635 INFO:teuthology.orchestra.run.smithi092.stdout:3494 write 0x15cbc18 thru 0x15d043f (0x4828 bytes) 2024-04-23T23:50:34.662 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:34.662 INFO:teuthology.orchestra.run.smithi092.stdout:3497 write 0x7ea0c58 thru 0x7eacb18 (0xbec1 bytes) 2024-04-23T23:50:34.668 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:34.681+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:34.672 INFO:teuthology.orchestra.run.smithi092.stdout:3499 read 0x976d06d thru 0x97723ed (0x5381 bytes) 2024-04-23T23:50:34.672 INFO:teuthology.orchestra.run.smithi092.stdout:3501 punch from 0x82c4ded to 0x82cb7d3, (0x69e6 bytes) 2024-04-23T23:50:34.672 INFO:teuthology.orchestra.run.smithi092.stdout:3502 writesame 0x8482bb2 thru 0x848bd1f data_size 0x1e(0x916e bytes) 2024-04-23T23:50:34.675 INFO:teuthology.orchestra.run.smithi092.stdout:3503 writesame 0x968c654 thru 0x96995a5 data_size 0x77(0xcf52 bytes) 2024-04-23T23:50:34.701 INFO:teuthology.orchestra.run.smithi092.stdout:3504 read 0xe673e23 thru 0xe6793dd (0x55bb bytes) 2024-04-23T23:50:34.706 INFO:teuthology.orchestra.run.smithi092.stdout:3505 trunc from 0xe86e473 to 0x9e1703e 2024-04-23T23:50:34.707 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:34.707 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:34.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:34.725 INFO:teuthology.orchestra.run.smithi092.stdout:3506 punch from 0x501bb44 to 0x502911e, (0xd5da bytes) 2024-04-23T23:50:34.726 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:34.726 INFO:teuthology.orchestra.run.smithi092.stdout:3508 punch from 0x83d8228 to 0x83dc718, (0x44f0 bytes) 2024-04-23T23:50:34.726 INFO:teuthology.orchestra.run.smithi092.stdout:3510 write 0x3c3ab71 thru 0x3c45807 (0xac97 bytes) 2024-04-23T23:50:34.737 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:34.737 INFO:teuthology.orchestra.run.smithi092.stdout:3513 write 0x4ea02aa thru 0x4eab710 (0xb467 bytes) 2024-04-23T23:50:34.765 INFO:teuthology.orchestra.run.smithi092.stdout:3514 writesame 0x88802ae thru 0x8889aa3 data_size 0x2(0x97f6 bytes) 2024-04-23T23:50:34.770 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:34.783+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:34.783 INFO:teuthology.orchestra.run.smithi092.stdout:3515 punch from 0x5517c46 to 0x551f35b, (0x7715 bytes) 2024-04-23T23:50:34.783 INFO:teuthology.orchestra.run.smithi092.stdout:3516 read 0x5935337 thru 0x593c0bf (0x6d89 bytes) 2024-04-23T23:50:34.783 INFO:teuthology.orchestra.run.smithi092.stdout:3518 write 0x881fb5f thru 0x8821317 (0x17b9 bytes) 2024-04-23T23:50:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:3520 read 0x6a69068 thru 0x6a6934a (0x2e3 bytes) 2024-04-23T23:50:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:3521 read 0x359c824 thru 0x35a24fe (0x5cdb bytes) 2024-04-23T23:50:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:3522 read 0x7082ebd thru 0x70890f2 (0x6236 bytes) 2024-04-23T23:50:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:3523 read 0x669c323 thru 0x66a56b8 (0x9396 bytes) 2024-04-23T23:50:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:3525 write 0x2fe24c4 thru 0x2fec5d4 (0xa111 bytes) 2024-04-23T23:50:34.830 INFO:teuthology.orchestra.run.smithi092.stdout:3526 writesame 0xaa43f6e thru 0xaa4caf6 data_size 0x3f(0x8b89 bytes) 2024-04-23T23:50:34.830 INFO:teuthology.orchestra.run.smithi092.stdout:3527 punch from 0x7f4a8b5 to 0x7f58f30, (0xe67b bytes) 2024-04-23T23:50:34.830 INFO:teuthology.orchestra.run.smithi092.stdout:3529 read 0x4d72f51 thru 0x4d79aea (0x6b9a bytes) 2024-04-23T23:50:34.830 INFO:teuthology.orchestra.run.smithi092.stdout:3530 writesame 0x2c71d2 thru 0x2cea83 data_size 0x7(0x78b2 bytes) 2024-04-23T23:50:34.869 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.766868831, 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:50:34.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:50:34.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:34.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7642745, 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:50:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:34.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:50:34.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.652619061, 2024-04-23T23:50:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:50:34.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:50:34.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65260062100000005, 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.887 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:50:34.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.892 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65258151499999995, 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:34.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:50:34.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:50:34.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65256300599999995, 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65245995700000003, 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:50:34.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:50:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65244033800000001, 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:34.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:50:34.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:50:34.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65233486299999999, 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:50:34.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:50:34.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:50:34.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:50:34.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:50:34.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:50:34.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:50:34.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:50:34.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.919 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.920 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:34.920 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65230967799999995, 2024-04-23T23:50:34.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:50:34.920 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:50:34.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:34.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:34.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:34.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:34.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:50:34.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:34.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:50:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:50:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:50:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:50:34.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:50:34.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.929 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65227613299999998, 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.930 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:50:34.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:50:34.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:50:34.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:50:34.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:50:34.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:50:34.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:50:34.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:50:34.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:50:34.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:50:34.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.939 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:34.940 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65225237000000003, 2024-04-23T23:50:34.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:50:34.940 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:34.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:34.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:34.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:50:34.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:50:34.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:50:34.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65223132900000003, 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65216309400000005, 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:50:34.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:50:34.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:50:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.958 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:34.958 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65213862600000005, 2024-04-23T23:50:34.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:50:34.958 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.958 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:50:34.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:34.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:34.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:50:34.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:50:34.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65211292799999998, 2024-04-23T23:50:34.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:50:34.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:50:34.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:50:34.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65209116199999995, 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65207069600000001, 2024-04-23T23:50:34.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:34.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:50:34.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65199283699999999, 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:34.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65196318600000003, 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:34.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:34.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:34.987 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:34.987 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:34.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3531 read 0x7bd88e7 thru 0x7bde9c8 (0x60e2 bytes) 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3533 trunc from 0xaa4caf7 to 0x78f8a39 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3534 write 0x2afcdcd thru 0x2b0532f (0x8563 bytes) 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3535 punch from 0x40b7a32 to 0x40c1e61, (0xa42f bytes) 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3536 write 0xa9c842b thru 0xa9d3a30 (0xb606 bytes) 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3538 writesame 0xa7d834f thru 0xa7dc0c8 data_size 0x6(0x3d7a bytes) 2024-04-23T23:50:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:3539 trunc from 0xa9d3a31 to 0x23731b 2024-04-23T23:50:34.990 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:35.002+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:35.037 INFO:teuthology.orchestra.run.smithi092.stdout:3540 write 0x7f9968e thru 0x7f9dd14 (0x4687 bytes) 2024-04-23T23:50:35.042 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:35.042 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:3545 write 0xacede27 thru 0xacf56ce (0x78a8 bytes) 2024-04-23T23:50:35.057 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:35.057 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:35.059 INFO:teuthology.orchestra.run.smithi092.stdout:3549 trunc from 0xacf56cf to 0x9059ce2 2024-04-23T23:50:35.068 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:35.068 INFO:teuthology.orchestra.run.smithi092.stdout:3551 punch from 0x3d93ca8 to 0x3da39e4, (0xfd3c bytes) 2024-04-23T23:50:35.068 INFO:teuthology.orchestra.run.smithi092.stdout:3553 write 0x6fa3b3e thru 0x6fa77d6 (0x3c99 bytes) 2024-04-23T23:50:35.073 INFO:teuthology.orchestra.run.smithi092.stdout:3554 read 0x3893551 thru 0x38a1776 (0xe226 bytes) 2024-04-23T23:50:35.073 INFO:teuthology.orchestra.run.smithi092.stdout:3556 write 0x1dc07eb thru 0x1dcc350 (0xbb66 bytes) 2024-04-23T23:50:35.083 INFO:teuthology.orchestra.run.smithi092.stdout:3558 write 0x2cc6d22 thru 0x2cd23a1 (0xb680 bytes) 2024-04-23T23:50:35.088 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:35.102+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:35.095 INFO:teuthology.orchestra.run.smithi092.stdout:3559 read 0x39b674a thru 0x39c3864 (0xd11b bytes) 2024-04-23T23:50:35.105 INFO:teuthology.orchestra.run.smithi092.stdout:3560 trunc from 0x9059ce2 to 0x14e2b48 2024-04-23T23:50:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:50:35.138 INFO:teuthology.orchestra.run.smithi092.stdout:3563 write 0xb4c6293 thru 0xb4cc3e7 (0x6155 bytes) 2024-04-23T23:50:35.149 INFO:teuthology.orchestra.run.smithi092.stdout:3564 read 0x7a32557 thru 0x7a41a82 (0xf52c bytes) 2024-04-23T23:50:35.149 INFO:teuthology.orchestra.run.smithi092.stdout:3565 write 0x5d5329b thru 0x5d6078e (0xd4f4 bytes) 2024-04-23T23:50:35.151 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:35.151 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:35.151 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:35.151 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:35.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:35.202 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:35.216+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:35.291 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:35.305+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:35.303 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:35.303 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:35.303 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:35.303 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:35.303 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:35.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:35.393 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:35.406+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:35.416 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:35.416 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:35.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:35.416 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.613322842000001, 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:50:35.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.611278859, 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:35.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.609301610999999, 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:35.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:50:35.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:50:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:35.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.609270978, 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:50:35.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.607328128000001, 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.605242919, 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:50:35.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.603233488000001, 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:50:35.444 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:50:35.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:50:35.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:50:35.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.601333520000001, 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:50:35.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.597284992000001, 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:50:35.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.595240316, 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:50:35.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.593166742999999, 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:50:35.462 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:50:35.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:50:35.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.591278587, 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.467 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:50:35.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:50:35.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.59125781, 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.471 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:50:35.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.583266316, 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.581331334, 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:50:35.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:35.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.581305239000001, 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:50:35.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:50:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1995877749999999, 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.489 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:50:35.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.199545356, 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:50:35.493 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:50:35.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:50:35.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1995093990000001, 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:50:35.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:50:35.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:50:35.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:35.502 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.199485041, 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:35.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:50:35.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.507 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:35.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:35.509 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:35.522+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:35.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:35.609 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:35.622+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:68261 3.19 3.199d02f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:35.167002+0000", 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.50245477900000002, 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50247540599999996, 2024-04-23T23:50:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for peered", 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68261 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167002+0000", 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167002+0000", 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167003+0000", 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167013+0000", 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0086e-05 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167024+0000", 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1375e-05 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167032+0000", 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8359999999999992e-06 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167088+0000", 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5847000000000002e-05 2024-04-23T23:50:35.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:35.167096+0000", 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0029999999999994e-06 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:35.659 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-04-23T23:50:35.660 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:35.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:35.669 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:35.687 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:35.688 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:35.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:35.710 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:35.723+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:35.722 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:35.739 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:35.757 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:35.757 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:35.773 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:35.773 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:35.774 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:35.775 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:35.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:35.792 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:35.792 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:35.810 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:35.824+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:35.894 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:35.894 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:35.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:35.912 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:35.926+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:36.007 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.007 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.012 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:36.026+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:36.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:36.113 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:36.127+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:36.129 INFO:teuthology.orchestra.run.smithi092.stdout:3567 read 0x2683654 thru 0x268d9fa (0xa3a7 bytes) 2024-04-23T23:50:36.129 INFO:teuthology.orchestra.run.smithi092.stdout:3568 write 0x952e776 thru 0x9532e91 (0x471c bytes) 2024-04-23T23:50:36.134 INFO:teuthology.orchestra.run.smithi092.stdout:3569 write 0x4afb16b thru 0x4afe668 (0x34fe bytes) 2024-04-23T23:50:36.138 INFO:teuthology.orchestra.run.smithi092.stdout:3571 punch from 0x3edd6a2 to 0x3ee1a12, (0x4370 bytes) 2024-04-23T23:50:36.138 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xedab073 2024-04-23T23:50:36.142 INFO:teuthology.orchestra.run.smithi092.stdout:3572 trunc from 0xb4cc3e8 to 0xedab073 2024-04-23T23:50:36.148 INFO:teuthology.orchestra.run.smithi092.stdout:3573 read 0xa0a490d thru 0xa0ad19c (0x8890 bytes) 2024-04-23T23:50:36.148 INFO:teuthology.orchestra.run.smithi092.stdout:3574 punch from 0x4536480 to 0x4539dc3, (0x3943 bytes) 2024-04-23T23:50:36.148 INFO:teuthology.orchestra.run.smithi092.stdout:3577 write 0xeb0898b thru 0xeb0b600 (0x2c76 bytes) 2024-04-23T23:50:36.152 INFO:teuthology.orchestra.run.smithi092.stdout:3578 writesame 0x866892f thru 0x8676fb0 data_size 0x7(0xe682 bytes) 2024-04-23T23:50:36.161 INFO:teuthology.orchestra.run.smithi092.stdout:3579 read 0xca218e1 thru 0xca2c11a (0xa83a bytes) 2024-04-23T23:50:36.161 INFO:teuthology.orchestra.run.smithi092.stdout:3581 read 0x32be53 thru 0x32ce19 (0xfc7 bytes) 2024-04-23T23:50:36.161 INFO:teuthology.orchestra.run.smithi092.stdout:3582 read 0x17f749e thru 0x17fb274 (0x3dd7 bytes) 2024-04-23T23:50:36.168 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.168 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.170 INFO:teuthology.orchestra.run.smithi092.stdout:3583 trunc from 0xedab073 to 0x77f2aef 2024-04-23T23:50:36.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:36.183 INFO:teuthology.orchestra.run.smithi092.stdout:3584 read 0x32bad6 thru 0x330c50 (0x517b bytes) 2024-04-23T23:50:36.192 INFO:teuthology.orchestra.run.smithi092.stdout:3586 write 0xaca2f83 thru 0xaca3118 (0x196 bytes) 2024-04-23T23:50:36.195 INFO:teuthology.orchestra.run.smithi092.stdout:3587 write 0xacfe94 thru 0xad83ad (0x851a bytes) 2024-04-23T23:50:36.202 INFO:teuthology.orchestra.run.smithi092.stdout:3590 trunc from 0xaca3119 to 0xad14cc1 2024-04-23T23:50:36.207 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.207 INFO:teuthology.orchestra.run.smithi092.stdout:3593 read 0x30bb91e thru 0x30c5e3e (0xa521 bytes) 2024-04-23T23:50:36.207 INFO:teuthology.orchestra.run.smithi092.stdout:3594 write 0xbc228c thru 0xbc25c7 (0x33c bytes) 2024-04-23T23:50:36.209 INFO:teuthology.orchestra.run.smithi092.stdout:3596 trunc from 0xad14cc1 to 0xb1caa62 2024-04-23T23:50:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:3597 punch from 0x1bead3d to 0x1bf4242, (0x9505 bytes) 2024-04-23T23:50:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:3598 read 0x3cecc0c thru 0x3cf5137 (0x852c bytes) 2024-04-23T23:50:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:3599 write 0x568645d thru 0x568d08c (0x6c30 bytes) 2024-04-23T23:50:36.215 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:36.228+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:36.226 INFO:teuthology.orchestra.run.smithi092.stdout:3600 write 0x2d07dc8 thru 0x2d15f3f (0xe178 bytes) 2024-04-23T23:50:36.241 INFO:teuthology.orchestra.run.smithi092.stdout:3601 write 0x3d3e9ed thru 0x3d42367 (0x397b bytes) 2024-04-23T23:50:36.249 INFO:teuthology.orchestra.run.smithi092.stdout:3603 punch from 0x76d802c to 0x76d96a7, (0x167b bytes) 2024-04-23T23:50:36.249 INFO:teuthology.orchestra.run.smithi092.stdout:3604 trunc from 0xb1caa62 to 0xae1049a 2024-04-23T23:50:36.255 INFO:teuthology.orchestra.run.smithi092.stdout:3607 write 0x9df1c79 thru 0x9df7f4f (0x62d7 bytes) 2024-04-23T23:50:36.269 INFO:teuthology.orchestra.run.smithi092.stdout:3608 write 0xcd8dac9 thru 0xcd9a1d1 (0xc709 bytes) 2024-04-23T23:50:36.277 INFO:teuthology.orchestra.run.smithi092.stdout:3609 write 0x8f7f934 thru 0x8f8e8b7 (0xef84 bytes) 2024-04-23T23:50:36.292 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.293 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.297 INFO:teuthology.orchestra.run.smithi092.stdout:3610 writesame 0xec91cb1 thru 0xec96d7d data_size 0x15(0x50cd bytes) 2024-04-23T23:50:36.297 INFO:teuthology.orchestra.run.smithi092.stdout:3612 read 0xa9b1791 thru 0xa9bab80 (0x93f0 bytes) 2024-04-23T23:50:36.297 INFO:teuthology.orchestra.run.smithi092.stdout:3614 punch from 0x76e00d5 to 0x76e58df, (0x580a bytes) 2024-04-23T23:50:36.297 INFO:teuthology.orchestra.run.smithi092.stdout:3617 punch from 0x7452ced to 0x745e872, (0xbb85 bytes) 2024-04-23T23:50:36.297 INFO:teuthology.orchestra.run.smithi092.stdout:3618 punch from 0x66f3b51 to 0x66f404f, (0x4fe bytes) 2024-04-23T23:50:36.297 INFO:teuthology.orchestra.run.smithi092.stdout:3619 write 0x829155b thru 0x829fc92 (0xe738 bytes) 2024-04-23T23:50:36.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:36.308 INFO:teuthology.orchestra.run.smithi092.stdout:3620 write 0x7eb9818 thru 0x7ec2804 (0x8fed bytes) 2024-04-23T23:50:36.311 INFO:teuthology.orchestra.run.smithi092.stdout:3621 read 0xc12e14f thru 0xc130eeb (0x2d9d bytes) 2024-04-23T23:50:36.312 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.321 INFO:teuthology.orchestra.run.smithi092.stdout:3623 trunc from 0xec96d7e to 0x6e1acd1 2024-04-23T23:50:36.340 INFO:teuthology.orchestra.run.smithi092.stdout:3624 read 0x6a3d605 thru 0x6a470b5 (0x9ab1 bytes) 2024-04-23T23:50:36.340 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.346 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:36.360+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:36.353 INFO:teuthology.orchestra.run.smithi092.stdout:3626 write 0xbd7f256 thru 0xbd8a563 (0xb30e bytes) 2024-04-23T23:50:36.362 INFO:teuthology.orchestra.run.smithi092.stdout:3628 write 0x78300b thru 0x786024 (0x301a bytes) 2024-04-23T23:50:36.367 INFO:teuthology.orchestra.run.smithi092.stdout:3632 trunc from 0xbd8a564 to 0x9be887c 2024-04-23T23:50:36.378 INFO:teuthology.orchestra.run.smithi092.stdout:3633 read 0x41c3b0f thru 0x41ce4e9 (0xa9db bytes) 2024-04-23T23:50:36.378 INFO:teuthology.orchestra.run.smithi092.stdout:3634 read 0x991a0e9 thru 0x991a8cc (0x7e4 bytes) 2024-04-23T23:50:36.378 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.386 INFO:teuthology.orchestra.run.smithi092.stdout:3638 trunc from 0x9be887c to 0x2f6d9e7 2024-04-23T23:50:36.401 INFO:teuthology.orchestra.run.smithi092.stdout:3640 punch from 0x2dc685c to 0x2dcb102, (0x48a6 bytes) 2024-04-23T23:50:36.401 INFO:teuthology.orchestra.run.smithi092.stdout:3642 read 0xdad8f9 thru 0xdbba33 (0xe13b bytes) 2024-04-23T23:50:36.401 INFO:teuthology.orchestra.run.smithi092.stdout:3643 writesame 0x290cac3 thru 0x2913298 data_size 0x1(0x67d6 bytes) 2024-04-23T23:50:36.415 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:36.429+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:36.417 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.418 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:36.433 INFO:teuthology.orchestra.run.smithi092.stdout:3644 write 0x9733643 thru 0x973d1ca (0x9b88 bytes) 2024-04-23T23:50:36.447 INFO:teuthology.orchestra.run.smithi092.stdout:3645 trunc from 0x973d1cb to 0x65eff8a 2024-04-23T23:50:36.458 INFO:teuthology.orchestra.run.smithi092.stdout:3646 read 0x1d0edd1 thru 0x1d1006f (0x129f bytes) 2024-04-23T23:50:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:3647 writesame 0xe3181ad thru 0xe31c23f data_size 0x1(0x4093 bytes) 2024-04-23T23:50:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:3648 read 0x6298a5e thru 0x62a12a1 (0x8844 bytes) 2024-04-23T23:50:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:3649 punch from 0xcd7ef4c to 0xcd8b4f9, (0xc5ad bytes) 2024-04-23T23:50:36.476 INFO:teuthology.orchestra.run.smithi092.stdout:3651 trunc from 0xe31c240 to 0xc0dc0e8 2024-04-23T23:50:36.481 INFO:teuthology.orchestra.run.smithi092.stdout:3653 read 0xb8a7d46 thru 0xb8b3d76 (0xc031 bytes) 2024-04-23T23:50:36.482 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.482 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.482 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.482 INFO:teuthology.orchestra.run.smithi092.stdout:3658 write 0x2be6d6e thru 0x2be9cd8 (0x2f6b bytes) 2024-04-23T23:50:36.484 INFO:teuthology.orchestra.run.smithi092.stdout:3659 writesame 0x881ca81 thru 0x882316e data_size 0x55(0x66ee bytes) 2024-04-23T23:50:36.490 INFO:teuthology.orchestra.run.smithi092.stdout:3663 writesame 0x6b2d7ff thru 0x6b3577d data_size 0x1(0x7f7f bytes) 2024-04-23T23:50:36.517 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:36.531+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:36.522 INFO:teuthology.orchestra.run.smithi092.stdout:3664 write 0xe0e2a9f thru 0xe0e6782 (0x3ce4 bytes) 2024-04-23T23:50:36.527 INFO:teuthology.orchestra.run.smithi092.stdout:3665 write 0x5913b28 thru 0x591557f (0x1a58 bytes) 2024-04-23T23:50:36.530 INFO:teuthology.orchestra.run.smithi092.stdout:3666 write 0x8e52750 thru 0x8e551dc (0x2a8d bytes) 2024-04-23T23:50:36.536 INFO:teuthology.orchestra.run.smithi092.stdout:3667 writesame 0x5818ebc thru 0x582289a data_size 0x1(0x99df bytes) 2024-04-23T23:50:36.571 INFO:teuthology.orchestra.run.smithi092.stdout:3668 write 0xee241d4 thru 0xee24309 (0x136 bytes) 2024-04-23T23:50:36.573 INFO:teuthology.orchestra.run.smithi092.stdout:3670 read 0x329e15d thru 0x32a7ede (0x9d82 bytes) 2024-04-23T23:50:36.573 INFO:teuthology.orchestra.run.smithi092.stdout:3671 read 0xdd56ef thru 0xdd89f4 (0x3306 bytes) 2024-04-23T23:50:36.574 INFO:teuthology.orchestra.run.smithi092.stdout:3672 punch from 0x1110d04 to 0x11149c3, (0x3cbf bytes) 2024-04-23T23:50:36.578 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.579 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.579 INFO:teuthology.orchestra.run.smithi092.stdout:3673 trunc from 0xee2430a to 0xa3943cd 2024-04-23T23:50:36.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:36.600 INFO:teuthology.orchestra.run.smithi092.stdout:3674 write 0xab3f39c thru 0xab405fc (0x1261 bytes) 2024-04-23T23:50:36.607 INFO:teuthology.orchestra.run.smithi092.stdout:3676 trunc from 0xab405fd to 0xb9fa9a2 2024-04-23T23:50:36.618 INFO:teuthology.orchestra.run.smithi092.stdout:3678 write 0xc555c5f thru 0xc563afa (0xde9c bytes) 2024-04-23T23:50:36.629 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:36.643+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:36.632 INFO:teuthology.orchestra.run.smithi092.stdout:3680 trunc from 0xc563afb to 0x93c09f9 2024-04-23T23:50:36.649 INFO:teuthology.orchestra.run.smithi092.stdout:3682 write 0xa169dbf thru 0xa179be4 (0xfe26 bytes) 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:3683 punch from 0x5635356 to 0x5642a32, (0xd6dc bytes) 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:3684 punch from 0x7053eaa to 0x705c97d, (0x8ad3 bytes) 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:3688 read 0x49317a2 thru 0x493ff6c (0xe7cb bytes) 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:3689 read 0x6ce8d3d thru 0x6cf2f19 (0xa1dd bytes) 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.659 INFO:teuthology.orchestra.run.smithi092.stdout:3692 read 0x7218c5f thru 0x721acb2 (0x2054 bytes) 2024-04-23T23:50:36.668 INFO:teuthology.orchestra.run.smithi092.stdout:3693 write 0xcf8496f thru 0xcf931fe (0xe890 bytes) 2024-04-23T23:50:36.677 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:36.678 INFO:teuthology.orchestra.run.smithi092.stdout:3695 read 0xb7709cc thru 0xb773ca4 (0x32d9 bytes) 2024-04-23T23:50:36.678 INFO:teuthology.orchestra.run.smithi092.stdout:3697 read 0xafb6846 thru 0xafbbd96 (0x5551 bytes) 2024-04-23T23:50:36.678 INFO:teuthology.orchestra.run.smithi092.stdout:3698 punch from 0xb109fa0 to 0xb117da0, (0xde00 bytes) 2024-04-23T23:50:36.678 INFO:teuthology.orchestra.run.smithi092.stdout:3699 writesame 0x997c311 thru 0x997dcad data_size 0x1(0x199d bytes) 2024-04-23T23:50:36.683 INFO:teuthology.orchestra.run.smithi092.stdout:3700 read 0xa8885c8 thru 0xa89433b (0xbd74 bytes) 2024-04-23T23:50:36.688 INFO:teuthology.orchestra.run.smithi092.stdout:3701 trunc from 0xcf931ff to 0x8abc475 2024-04-23T23:50:36.702 INFO:teuthology.orchestra.run.smithi092.stdout:3702 clone 26 order 25 su 8388608 sc 8 2024-04-23T23:50:36.719 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:36.733+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:36.742 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.742 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:36.820 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:36.834+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:36.853 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:36.853 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:36.857 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:50:36.857 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:36.858 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-23T23:50:36.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-23T23:50:36.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:36.921 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:36.935+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:37.005 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.005 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.013 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:37.057 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:37.071+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:37.116 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.116 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.122 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:37.136+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:37.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:37.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 61, 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "6a51c05a-822c-4999-80c7-e8408cd2a202", 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-04-23T23:49:23.155452+0000", 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-04-23T23:50:37.136659+0000", 2024-04-23T23:50:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-04-23T23:50:32.112122+0000", 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-04-23T23:49:26.183177+0000", 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5799936, 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 6, 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-23T23:50:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 3, 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "jewel", 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "squid", 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_crimson": false, 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:35.387984+0000", 2024-04-23T23:50:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 1, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 1, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 1, 2024-04-23T23:50:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 1, 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 1, 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "18", 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:50:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:50:37.151 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:50:37.152 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:50:37.153 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 8, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 8, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.25, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 2, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 2, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-04-23T23:50:37.154 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "rbd", 2024-04-23T23:50:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:41.909447+0000", 2024-04-23T23:50:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-04-23T23:50:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-04-23T23:50:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:50:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 8, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 8, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 8, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 8, 2024-04-23T23:50:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 8, 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "21", 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:50:37.157 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:50:37.158 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:50:37.159 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-04-23T23:50:37.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.9900000095367432, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.9900000095367432, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.87999999523162842, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.75, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.75, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:50:37.161 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "pool_client.0", 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:49.482932+0000", 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:50:37.162 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-04-23T23:50:37.163 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "61", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 28, 2024-04-23T23:50:37.164 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 61, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:50:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:50:37.166 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:50:37.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.5, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.5, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 1, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.5, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.5, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.168 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "6a84eb3e-7d8a-4436-956d-f7da68d06a21", 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 58, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.169 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.170 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6820/3364976130", 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6824/3364976130", 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3364976130", 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6828/3364976130", 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-04-23T23:50:37.174 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "be35c480-9a84-4846-9f0b-043c7c2c9f4a", 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 59, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.175 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.176 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-04-23T23:50:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.179 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6817/2924722109", 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/2924722109", 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6829/2924722109", 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6825/2924722109", 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "64e8bbbb-690a-4267-a030-30ec66f30bfa", 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 56, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.182 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.183 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.184 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6801/3952319845", 2024-04-23T23:50:37.185 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6803/3952319845", 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3952319845", 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3952319845", 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "692beda9-1302-4d96-81ab-62c40a12e3f1", 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.186 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 15, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 38, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 56, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 59, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 39, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.187 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.188 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6809/1499077312", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6811/1499077312", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1499077312", 2024-04-23T23:50:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1499077312", 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "705f6b1f-c35d-4ed8-a2c9-70b9324691e3", 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.192 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 22, 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6824", 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6825", 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6826", 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6827", 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6830", 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6831", 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6828", 2024-04-23T23:50:37.196 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6829", 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6825/1281396523", 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6827/1281396523", 2024-04-23T23:50:37.197 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1281396523", 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1281396523", 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "eb37446d-b5c9-4200-a1e9-38c1f6268401", 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.198 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 59, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6800", 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.199 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6801", 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6802", 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6803", 2024-04-23T23:50:37.200 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6806", 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.201 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6807", 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6804", 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6805", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6801/612886539", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6803/612886539", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6807/612886539", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6805/612886539", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "c38a4da5-1017-4bb9-86ab-999ea57999c9", 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 58, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.204 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6816", 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6817", 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6818", 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6819", 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6822", 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6823", 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6820", 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6821", 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.208 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6817/2243867625", 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6819/2243867625", 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6823/2243867625", 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6821/2243867625", 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "a921d465-920a-4ad0-839f-d97cd5e6ce62", 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:50:37.209 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 58, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6808", 2024-04-23T23:50:37.210 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6809", 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6810", 2024-04-23T23:50:37.211 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6811", 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6814", 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6815", 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6812", 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6813", 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6809/3209176445", 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6811/3209176445", 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3209176445", 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3209176445", 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:50:37.214 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.405502+0000", 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.334094+0000", 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-04-23T23:50:37.216 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.268468+0000", 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.217 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.367807+0000", 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 39 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.218 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.328752+0000", 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.255936+0000", 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.219 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.298943+0000", 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:50:37.220 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.352632+0000", 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:37.221 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [], 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_primaries": [], 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": {}, 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-04-23T23:50:37.222 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [], 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:37.223 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:37.226 INFO:tasks.thrashosds.thrasher:j is {'epoch': 61, 'fsid': '6a51c05a-822c-4999-80c7-e8408cd2a202', 'created': '2024-04-23T23:49:23.155452+0000', 'modified': '2024-04-23T23:50:37.136659+0000', 'last_up_change': '2024-04-23T23:50:32.112122+0000', 'last_in_change': '2024-04-23T23:49:26.183177+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T23:49:35.387984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-23T23:49:41.909447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-04-23T23:49:49.482932+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6a84eb3e-7d8a-4436-956d-f7da68d06a21', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6820', 'nonce': 3364976130}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6824', 'nonce': 3364976130}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3364976130}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6828', 'nonce': 3364976130}]}, 'public_addr': '172.21.15.92:6820/3364976130', 'cluster_addr': '172.21.15.92:6824/3364976130', 'heartbeat_back_addr': '172.21.15.92:6831/3364976130', 'heartbeat_front_addr': '172.21.15.92:6828/3364976130', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'be35c480-9a84-4846-9f0b-043c7c2c9f4a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2924722109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6819', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 2924722109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6827', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 2924722109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6823', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 2924722109}]}, 'public_addr': '172.21.15.92:6817/2924722109', 'cluster_addr': '172.21.15.92:6821/2924722109', 'heartbeat_back_addr': '172.21.15.92:6829/2924722109', 'heartbeat_front_addr': '172.21.15.92:6825/2924722109', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64e8bbbb-690a-4267-a030-30ec66f30bfa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3952319845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3952319845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3952319845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3952319845}]}, 'public_addr': '172.21.15.92:6801/3952319845', 'cluster_addr': '172.21.15.92:6803/3952319845', 'heartbeat_back_addr': '172.21.15.92:6807/3952319845', 'heartbeat_front_addr': '172.21.15.92:6805/3952319845', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '692beda9-1302-4d96-81ab-62c40a12e3f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 38, 'up_from': 56, 'up_thru': 59, 'down_at': 39, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1499077312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1499077312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1499077312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1499077312}]}, 'public_addr': '172.21.15.92:6809/1499077312', 'cluster_addr': '172.21.15.92:6811/1499077312', 'heartbeat_back_addr': '172.21.15.92:6815/1499077312', 'heartbeat_front_addr': '172.21.15.92:6813/1499077312', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '705f6b1f-c35d-4ed8-a2c9-70b9324691e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1281396523}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1281396523}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1281396523}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1281396523}]}, 'public_addr': '172.21.15.119:6825/1281396523', 'cluster_addr': '172.21.15.119:6827/1281396523', 'heartbeat_back_addr': '172.21.15.119:6831/1281396523', 'heartbeat_front_addr': '172.21.15.119:6829/1281396523', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb37446d-b5c9-4200-a1e9-38c1f6268401', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 612886539}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 612886539}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 612886539}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 612886539}]}, 'public_addr': '172.21.15.119:6801/612886539', 'cluster_addr': '172.21.15.119:6803/612886539', 'heartbeat_back_addr': '172.21.15.119:6807/612886539', 'heartbeat_front_addr': '172.21.15.119:6805/612886539', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c38a4da5-1017-4bb9-86ab-999ea57999c9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 2243867625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 2243867625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 2243867625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 2243867625}]}, 'public_addr': '172.21.15.119:6817/2243867625', 'cluster_addr': '172.21.15.119:6819/2243867625', 'heartbeat_back_addr': '172.21.15.119:6823/2243867625', 'heartbeat_front_addr': '172.21.15.119:6821/2243867625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a921d465-920a-4ad0-839f-d97cd5e6ce62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3209176445}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3209176445}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3209176445}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3209176445}]}, 'public_addr': '172.21.15.119:6809/3209176445', 'cluster_addr': '172.21.15.119:6811/3209176445', 'heartbeat_back_addr': '172.21.15.119:6815/3209176445', 'heartbeat_front_addr': '172.21.15.119:6813/3209176445', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.405502+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.334094+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.268468+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.367807+0000', 'dead_epoch': 39}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.328752+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.255936+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.298943+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.352632+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-23T23:50:37.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:50:37.227 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone25 from 0x8abc475 (overlap 0x23731b) to 0x18d092 2024-04-23T23:50:37.227 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:37.238+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:37.296 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-04-23T23:50:37.311 INFO:teuthology.orchestra.run.smithi092.stdout:3703 write 0xd1be9bd thru 0xd1c6c50 (0x8294 bytes) 2024-04-23T23:50:37.315 INFO:teuthology.orchestra.run.smithi092.stdout:3705 read 0x190c5d3 thru 0x1913d5f (0x778d bytes) 2024-04-23T23:50:37.315 INFO:teuthology.orchestra.run.smithi092.stdout:3710 read 0x2632e39 thru 0x263c137 (0x92ff bytes) 2024-04-23T23:50:37.315 INFO:teuthology.orchestra.run.smithi092.stdout:3711 punch from 0x36fc96d to 0x3707a72, (0xb105 bytes) 2024-04-23T23:50:37.315 INFO:teuthology.orchestra.run.smithi092.stdout:3712 write 0xad0e51f thru 0xad1b4a9 (0xcf8b bytes) 2024-04-23T23:50:37.321 INFO:teuthology.orchestra.run.smithi092.stdout:3714 punch from 0x846c to 0x135b3, (0xb147 bytes) 2024-04-23T23:50:37.321 INFO:teuthology.orchestra.run.smithi092.stdout:3715 write 0x37bce4b thru 0x37c1237 (0x43ed bytes) 2024-04-23T23:50:37.325 INFO:teuthology.orchestra.run.smithi092.stdout:3717 write 0x5f5b405 thru 0x5f5ea7b (0x3677 bytes) 2024-04-23T23:50:37.325 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:37.339+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:37.333 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.333 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.336 INFO:teuthology.orchestra.run.smithi092.stdout:3718 trunc from 0xd1c6c51 to 0x6fc6288 2024-04-23T23:50:37.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:37.359 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.360 INFO:teuthology.orchestra.run.smithi092.stdout:3722 read 0x6a4ec07 thru 0x6a508ac (0x1ca6 bytes) 2024-04-23T23:50:37.369 INFO:teuthology.orchestra.run.smithi092.stdout:3723 write 0xa0863b2 thru 0xa089854 (0x34a3 bytes) 2024-04-23T23:50:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:3725 punch from 0x5fbc8f6 to 0x5fbd039, (0x743 bytes) 2024-04-23T23:50:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:3726 writesame 0x8312071 thru 0x8321c69 data_size 0x5f(0xfbf9 bytes) 2024-04-23T23:50:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:3727 read 0x16289df thru 0x16361ca (0xd7ec bytes) 2024-04-23T23:50:37.373 INFO:teuthology.orchestra.run.smithi092.stdout:3728 read 0x8f6b8e0 thru 0x8f7b122 (0xf843 bytes) 2024-04-23T23:50:37.377 INFO:teuthology.orchestra.run.smithi092.stdout:3729 trunc from 0xa089855 to 0xd7117d1 2024-04-23T23:50:37.382 INFO:teuthology.orchestra.run.smithi092.stdout:3730 read 0xa1124ed thru 0xa11b44d (0x8f61 bytes) 2024-04-23T23:50:37.383 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.389 INFO:teuthology.orchestra.run.smithi092.stdout:3732 write 0xe68ebf8 thru 0xe690a56 (0x1e5f bytes) 2024-04-23T23:50:37.396 INFO:teuthology.orchestra.run.smithi092.stdout:3733 trunc from 0xe690a57 to 0x9951411 2024-04-23T23:50:37.403 INFO:teuthology.orchestra.run.smithi092.stdout:3735 read 0x1122551 thru 0x1124f6c (0x2a1c bytes) 2024-04-23T23:50:37.415 INFO:teuthology.orchestra.run.smithi092.stdout:3738 writesame 0xe6a431f thru 0xe6a594f data_size 0x13(0x1631 bytes) 2024-04-23T23:50:37.416 INFO:teuthology.orchestra.run.smithi092.stdout:3739 write 0x7265207 thru 0x72684b9 (0x32b3 bytes) 2024-04-23T23:50:37.418 INFO:teuthology.orchestra.run.smithi092.stdout:3740 writesame 0xe43e114 thru 0xe4439eb data_size 0x4(0x58d8 bytes) 2024-04-23T23:50:37.423 INFO:teuthology.orchestra.run.smithi092.stdout:3741 write 0x73f6b71 thru 0x7401d59 (0xb1e9 bytes) 2024-04-23T23:50:37.424 INFO:teuthology.orchestra.run.smithi092.stdout:3742 read 0x94f3eaf thru 0x94fc50d (0x865f bytes) 2024-04-23T23:50:37.425 INFO:teuthology.orchestra.run.smithi092.stdout:3743 write 0x95863ab thru 0x958a9b5 (0x460b bytes) 2024-04-23T23:50:37.426 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:37.439+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:37.426 INFO:teuthology.orchestra.run.smithi092.stdout:3745 write 0xb286d84 thru 0xb29181e (0xaa9b bytes) 2024-04-23T23:50:37.428 INFO:teuthology.orchestra.run.smithi092.stdout:3747 write 0x9f7b328 thru 0x9f81a1e (0x66f7 bytes) 2024-04-23T23:50:37.429 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.429 INFO:teuthology.orchestra.run.smithi092.stdout:3750 write 0x1e140d2 thru 0x1e19bdb (0x5b0a bytes) 2024-04-23T23:50:37.437 INFO:teuthology.orchestra.run.smithi092.stdout:3751 trunc from 0xe6a5950 to 0x95d618d 2024-04-23T23:50:37.444 INFO:teuthology.orchestra.run.smithi092.stdout:3752 write 0x8734308 thru 0x8739549 (0x5242 bytes) 2024-04-23T23:50:37.446 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.451 INFO:teuthology.orchestra.run.smithi092.stdout:3755 trunc from 0x95d618d to 0x528a10f 2024-04-23T23:50:37.457 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.457 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:37.471 INFO:teuthology.orchestra.run.smithi092.stdout:3757 write 0xa0fad5a thru 0xa0fda19 (0x2cc0 bytes) 2024-04-23T23:50:37.473 INFO:teuthology.orchestra.run.smithi092.stdout:3760 write 0x769b44e thru 0x769c78b (0x133e bytes) 2024-04-23T23:50:37.483 INFO:teuthology.orchestra.run.smithi092.stdout:3761 write 0xca997f4 thru 0xcaa18c5 (0x80d2 bytes) 2024-04-23T23:50:37.484 INFO:teuthology.orchestra.run.smithi092.stdout:3762 read 0x3f451c3 thru 0x3f4f042 (0x9e80 bytes) 2024-04-23T23:50:37.485 INFO:teuthology.orchestra.run.smithi092.stdout:3765 punch from 0x879bbc5 to 0x87aa254, (0xe68f bytes) 2024-04-23T23:50:37.485 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.485 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.485 INFO:teuthology.orchestra.run.smithi092.stdout:3768 read 0x162469e thru 0x1626700 (0x2063 bytes) 2024-04-23T23:50:37.485 INFO:teuthology.orchestra.run.smithi092.stdout:3769 read 0x589b22e thru 0x58a5086 (0x9e59 bytes) 2024-04-23T23:50:37.485 INFO:teuthology.orchestra.run.smithi092.stdout:3771 write 0x549457c thru 0x5497442 (0x2ec7 bytes) 2024-04-23T23:50:37.493 INFO:teuthology.orchestra.run.smithi092.stdout:3772 write 0xd1e4662 thru 0xd1ea128 (0x5ac7 bytes) 2024-04-23T23:50:37.494 INFO:teuthology.orchestra.run.smithi092.stdout:3773 read 0x8cd2faf thru 0x8ce15e4 (0xe636 bytes) 2024-04-23T23:50:37.495 INFO:teuthology.orchestra.run.smithi092.stdout:3774 read 0x505b91e thru 0x505b9f9 (0xdc bytes) 2024-04-23T23:50:37.495 INFO:teuthology.orchestra.run.smithi092.stdout:3775 write 0xc959177 thru 0xc963c91 (0xab1b bytes) 2024-04-23T23:50:37.497 INFO:teuthology.orchestra.run.smithi092.stdout:3778 write 0xad69234 thru 0xad76705 (0xd4d2 bytes) 2024-04-23T23:50:37.498 INFO:teuthology.orchestra.run.smithi092.stdout:3779 read 0xa9ef4dd thru 0xa9f1c24 (0x2748 bytes) 2024-04-23T23:50:37.499 INFO:teuthology.orchestra.run.smithi092.stdout:3780 read 0x757ef8 thru 0x7652aa (0xd3b3 bytes) 2024-04-23T23:50:37.503 INFO:teuthology.orchestra.run.smithi092.stdout:3781 trunc from 0xd1ea129 to 0xa0e94b6 2024-04-23T23:50:37.509 INFO:teuthology.orchestra.run.smithi092.stdout:3782 read 0x2a45fa1 thru 0x2a4c50f (0x656f bytes) 2024-04-23T23:50:37.510 INFO:teuthology.orchestra.run.smithi092.stdout:3783 writesame 0x844ca35 thru 0x84544f8 data_size 0x6c(0x7ac4 bytes) 2024-04-23T23:50:37.511 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.511 INFO:teuthology.orchestra.run.smithi092.stdout:3786 punch from 0x849643a to 0x84a11c2, (0xad88 bytes) 2024-04-23T23:50:37.511 INFO:teuthology.orchestra.run.smithi092.stdout:3787 read 0x220cfe9 thru 0x2217748 (0xa760 bytes) 2024-04-23T23:50:37.512 INFO:teuthology.orchestra.run.smithi092.stdout:3789 writesame 0x4c88407 thru 0x4c89db7 data_size 0x1(0x19b1 bytes) 2024-04-23T23:50:37.517 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.518 INFO:teuthology.orchestra.run.smithi092.stdout:3792 trunc from 0xa0e94b6 to 0xafd8882 2024-04-23T23:50:37.523 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.523 INFO:teuthology.orchestra.run.smithi092.stdout:3795 read 0x5a9e5e0 thru 0x5aa807d (0x9a9e bytes) 2024-04-23T23:50:37.524 INFO:teuthology.orchestra.run.smithi092.stdout:3797 punch from 0x44805e3 to 0x448d12e, (0xcb4b bytes) 2024-04-23T23:50:37.524 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.524 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.524 INFO:teuthology.orchestra.run.smithi092.stdout:3800 read 0xaa3880e thru 0xaa3db6b (0x535e bytes) 2024-04-23T23:50:37.524 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.525 INFO:teuthology.orchestra.run.smithi092.stdout:3802 trunc from 0xafd8882 to 0x9c4cf63 2024-04-23T23:50:37.526 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:37.540+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:37.532 INFO:teuthology.orchestra.run.smithi092.stdout:3806 writesame 0xb4e07c thru 0xb4f467 data_size 0x19(0x13ec bytes) 2024-04-23T23:50:37.533 INFO:teuthology.orchestra.run.smithi092.stdout:3807 read 0x8eb9fa7 thru 0x8ebcad3 (0x2b2d bytes) 2024-04-23T23:50:37.534 INFO:teuthology.orchestra.run.smithi092.stdout:3810 punch from 0x123baa6 to 0x1244725, (0x8c7f bytes) 2024-04-23T23:50:37.534 INFO:teuthology.orchestra.run.smithi092.stdout:3811 read 0x57a431a thru 0x57ab425 (0x710c bytes) 2024-04-23T23:50:37.535 INFO:teuthology.orchestra.run.smithi092.stdout:3812 trunc from 0x9c4cf63 to 0x956d465 2024-04-23T23:50:37.541 INFO:teuthology.orchestra.run.smithi092.stdout:3813 write 0x512a27a thru 0x512d029 (0x2db0 bytes) 2024-04-23T23:50:37.543 INFO:teuthology.orchestra.run.smithi092.stdout:3814 read 0x5952616 thru 0x5961c5c (0xf647 bytes) 2024-04-23T23:50:37.546 INFO:teuthology.orchestra.run.smithi092.stdout:3815 trunc from 0x956d465 to 0x6fc78db 2024-04-23T23:50:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:3816 write 0x1ad5671 thru 0x1ade950 (0x92e0 bytes) 2024-04-23T23:50:37.565 INFO:teuthology.orchestra.run.smithi092.stdout:3818 write 0xb15e735 thru 0xb16068c (0x1f58 bytes) 2024-04-23T23:50:37.573 INFO:teuthology.orchestra.run.smithi092.stdout:3819 write 0xb937cb3 thru 0xb9454ae (0xd7fc bytes) 2024-04-23T23:50:37.581 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.581 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.582 INFO:teuthology.orchestra.run.smithi092.stdout:3820 write 0xd2cc302 thru 0xd2d0d5a (0x4a59 bytes) 2024-04-23T23:50:37.583 INFO:teuthology.orchestra.run.smithi092.stdout:3821 write 0xb7c5074 thru 0xb7c8d93 (0x3d20 bytes) 2024-04-23T23:50:37.585 INFO:teuthology.orchestra.run.smithi092.stdout:3822 writesame 0x86c6ace thru 0x86d15a1 data_size 0x3a(0xaad4 bytes) 2024-04-23T23:50:37.587 INFO:teuthology.orchestra.run.smithi092.stdout:3823 read 0x8f6eb7e thru 0x8f72005 (0x3488 bytes) 2024-04-23T23:50:37.587 INFO:teuthology.orchestra.run.smithi092.stdout:3824 read 0x377ca6a thru 0x3782a01 (0x5f98 bytes) 2024-04-23T23:50:37.588 INFO:teuthology.orchestra.run.smithi092.stdout:3826 writesame 0x3674c3d thru 0x367d024 data_size 0x54(0x83e8 bytes) 2024-04-23T23:50:37.589 INFO:teuthology.orchestra.run.smithi092.stdout:3828 write 0x7ea942c thru 0x7eadb8f (0x4764 bytes) 2024-04-23T23:50:37.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:50:37.594 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:50:37.598 INFO:teuthology.orchestra.run.smithi092.stdout:3829 write 0xdd62988 thru 0xdd63dff (0x1478 bytes) 2024-04-23T23:50:37.599 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.600 INFO:teuthology.orchestra.run.smithi092.stdout:3831 read 0xc4c3641 thru 0xc4c3dac (0x76c bytes) 2024-04-23T23:50:37.600 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.600 INFO:teuthology.orchestra.run.smithi092.stdout:3837 writesame 0x3e75304 thru 0x3e7b386 data_size 0x1(0x6083 bytes) 2024-04-23T23:50:37.605 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-04-23T23:50:37.366543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":908308364,"num_objects":1186,"num_object_clones":291,"num_object_copies":2372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1186,"num_whiteouts":480,"num_read":82585,"num_read_kb":323891,"num_write":9026,"num_write_kb":190097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":12091085,"num_keys_recovered":59,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8050,"ondisk_log_size":8050,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466568,"kb_used_data":194692,"kb_used_omap":21,"kb_used_meta":271658,"kb_avail":754508152,"statfs":{"total":773094113280,"available":772616347648,"internally_reserved":0,"allocated":199364608,"data_stored":256707107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":278178165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38862417,"num_objects":413,"num_object_clones":102,"num_object_copies":826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413,"num_whiteouts":234,"num_read":10168,"num_read_kb":55537,"num_write":1450,"num_write_kb":23947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":12091085,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000667"},"pg_stats":[{"pgid":"3.1f","version":"58'438","reported_seq":6802,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.135976+0000","last_change":"2024-04-23T23:49:53.323402+0000","last_active":"2024-04-23T23:50:35.135976+0000","last_peered":"2024-04-23T23:50:35.135976+0000","last_clean":"2024-04-23T23:50:35.135976+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:50:35.135976+0000","last_undegraded":"2024-04-23T23:50:35.135976+0000","last_fullsized":"2024-04-23T23:50:35.135976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:39:08.751962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085782299999999996,"stat_sum":{"num_bytes":14522786,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":8381,"num_read_kb":21127,"num_write":604,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"58'432","reported_seq":1259,"reported_epoch":59,"state":"peering","last_fresh":"2024-04-23T23:50:35.139339+0000","last_change":"2024-04-23T23:50:35.139107+0000","last_active":"2024-04-23T23:50:35.139107+0000","last_peered":"2024-04-23T23:50:34.976022+0000","last_clean":"2024-04-23T23:50:07.499512+0000","last_became_active":"2024-04-23T23:50:34.091249+0000","last_became_peered":"2024-04-23T23:50:34.091249+0000","last_unstale":"2024-04-23T23:50:35.139339+0000","last_undegraded":"2024-04-23T23:50:35.139339+0000","last_fullsized":"2024-04-23T23:50:35.139339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067565599999999996,"stat_sum":{"num_bytes":41679679,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2154,"num_read_kb":3335,"num_write":500,"num_write_kb":3169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"55'133","reported_seq":2470,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.952972+0000","last_change":"2024-04-23T23:49:53.328510+0000","last_active":"2024-04-23T23:50:33.952972+0000","last_peered":"2024-04-23T23:50:33.952972+0000","last_clean":"2024-04-23T23:50:33.952972+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:50:33.952972+0000","last_undegraded":"2024-04-23T23:50:33.952972+0000","last_fullsized":"2024-04-23T23:50:33.952972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:45.201819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018779,"stat_sum":{"num_bytes":10772578,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":2459,"num_read_kb":7194,"num_write":144,"num_write_kb":1158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"58'261","reported_seq":1278,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.360584+0000","last_change":"2024-04-23T23:49:53.328456+0000","last_active":"2024-04-23T23:50:34.360584+0000","last_peered":"2024-04-23T23:50:34.360584+0000","last_clean":"2024-04-23T23:50:34.360584+0000","last_became_active":"2024-04-23T23:49:51.317767+0000","last_became_peered":"2024-04-23T23:49:51.317767+0000","last_unstale":"2024-04-23T23:50:34.360584+0000","last_undegraded":"2024-04-23T23:50:34.360584+0000","last_fullsized":"2024-04-23T23:50:34.360584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:28.344671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099769099999999999,"stat_sum":{"num_bytes":19414217,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1943,"num_read_kb":5834,"num_write":283,"num_write_kb":5522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"58'165","reported_seq":2217,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.136246+0000","last_change":"2024-04-23T23:49:53.322810+0000","last_active":"2024-04-23T23:50:35.136246+0000","last_peered":"2024-04-23T23:50:35.136246+0000","last_clean":"2024-04-23T23:50:35.136246+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:50:35.136246+0000","last_undegraded":"2024-04-23T23:50:35.136246+0000","last_fullsized":"2024-04-23T23:50:35.136246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:40.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079418399999999997,"stat_sum":{"num_bytes":48980506,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":2554,"num_read_kb":14178,"num_write":143,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"58'131","reported_seq":1503,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.360367+0000","last_change":"2024-04-23T23:49:53.322527+0000","last_active":"2024-04-23T23:50:34.360367+0000","last_peered":"2024-04-23T23:50:34.360367+0000","last_clean":"2024-04-23T23:50:34.360367+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:50:34.360367+0000","last_undegraded":"2024-04-23T23:50:34.360367+0000","last_fullsized":"2024-04-23T23:50:34.360367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:52.942998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041081400000000001,"stat_sum":{"num_bytes":6914834,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":1540,"num_read_kb":3228,"num_write":153,"num_write_kb":937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"58'631","reported_seq":4449,"reported_epoch":58,"state":"peering","last_fresh":"2024-04-23T23:50:35.007589+0000","last_change":"2024-04-23T23:50:35.140490+0000","last_active":"2024-04-23T23:50:35.140490+0000","last_peered":"2024-04-23T23:50:35.007589+0000","last_clean":"2024-04-23T23:50:09.586386+0000","last_became_active":"2024-04-23T23:50:34.091825+0000","last_became_peered":"2024-04-23T23:50:34.091825+0000","last_unstale":"2024-04-23T23:50:35.007589+0000","last_undegraded":"2024-04-23T23:50:35.007589+0000","last_fullsized":"2024-04-23T23:50:33.950556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038598300000000001,"stat_sum":{"num_bytes":9340518,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":6405,"num_read_kb":9653,"num_write":921,"num_write_kb":2247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":371151,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"58'254","reported_seq":1003,"reported_epoch":59,"state":"peering","last_fresh":"2024-04-23T23:50:35.139260+0000","last_change":"2024-04-23T23:50:35.139159+0000","last_active":"2024-04-23T23:50:35.139159+0000","last_peered":"2024-04-23T23:50:34.474762+0000","last_clean":"2024-04-23T23:50:09.480745+0000","last_became_active":"2024-04-23T23:50:34.089933+0000","last_became_peered":"2024-04-23T23:50:34.089933+0000","last_unstale":"2024-04-23T23:50:35.139260+0000","last_undegraded":"2024-04-23T23:50:35.139260+0000","last_fullsized":"2024-04-23T23:50:35.139260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064860299999999998,"stat_sum":{"num_bytes":22960515,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":1178,"num_read_kb":5282,"num_write":298,"num_write_kb":3339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"58'322","reported_seq":1675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.136090+0000","last_change":"2024-04-23T23:49:53.323222+0000","last_active":"2024-04-23T23:50:35.136090+0000","last_peered":"2024-04-23T23:50:35.136090+0000","last_clean":"2024-04-23T23:50:35.136090+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:50:35.136090+0000","last_undegraded":"2024-04-23T23:50:35.136090+0000","last_fullsized":"2024-04-23T23:50:35.136090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:05:39.120649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091814399999999997,"stat_sum":{"num_bytes":56423510,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":1894,"num_read_kb":11998,"num_write":391,"num_write_kb":8831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"58'158","reported_seq":3358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.380061+0000","last_change":"2024-04-23T23:49:53.327873+0000","last_active":"2024-04-23T23:50:34.380061+0000","last_peered":"2024-04-23T23:50:34.380061+0000","last_clean":"2024-04-23T23:50:34.380061+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:50:34.380061+0000","last_undegraded":"2024-04-23T23:50:34.380061+0000","last_fullsized":"2024-04-23T23:50:34.380061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:45.774458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039462200000000002,"stat_sum":{"num_bytes":54186873,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":3184,"num_read_kb":27405,"num_write":126,"num_write_kb":15331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"58'155","reported_seq":1149,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.370265+0000","last_change":"2024-04-23T23:50:33.039363+0000","last_active":"2024-04-23T23:50:34.370265+0000","last_peered":"2024-04-23T23:50:34.370265+0000","last_clean":"2024-04-23T23:50:34.370265+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:50:34.370265+0000","last_undegraded":"2024-04-23T23:50:34.370265+0000","last_fullsized":"2024-04-23T23:50:34.370265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:41.243015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066676300000000001,"stat_sum":{"num_bytes":30159229,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":441,"num_read_kb":5955,"num_write":158,"num_write_kb":8263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"58'138","reported_seq":2918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.380183+0000","last_change":"2024-04-23T23:49:53.322838+0000","last_active":"2024-04-23T23:50:34.380183+0000","last_peered":"2024-04-23T23:50:34.380183+0000","last_clean":"2024-04-23T23:50:34.380183+0000","last_became_active":"2024-04-23T23:49:51.320365+0000","last_became_peered":"2024-04-23T23:49:51.320365+0000","last_unstale":"2024-04-23T23:50:34.380183+0000","last_undegraded":"2024-04-23T23:50:34.380183+0000","last_fullsized":"2024-04-23T23:50:34.380183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:43:56.947161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042792500000000001,"stat_sum":{"num_bytes":32295595,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2724,"num_read_kb":12424,"num_write":129,"num_write_kb":15636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.952784+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:50:33.952784+0000","last_peered":"2024-04-23T23:50:33.952784+0000","last_clean":"2024-04-23T23:50:33.952784+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:50:33.952784+0000","last_undegraded":"2024-04-23T23:50:33.952784+0000","last_fullsized":"2024-04-23T23:50:33.952784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"58'397","reported_seq":1622,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.136191+0000","last_change":"2024-04-23T23:49:53.322476+0000","last_active":"2024-04-23T23:50:35.136191+0000","last_peered":"2024-04-23T23:50:35.136191+0000","last_clean":"2024-04-23T23:50:35.136191+0000","last_became_active":"2024-04-23T23:49:51.317534+0000","last_became_peered":"2024-04-23T23:49:51.317534+0000","last_unstale":"2024-04-23T23:50:35.136191+0000","last_undegraded":"2024-04-23T23:50:35.136191+0000","last_fullsized":"2024-04-23T23:50:35.136191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:11:28.882998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046868700000000002,"stat_sum":{"num_bytes":29928795,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":2553,"num_read_kb":8719,"num_write":534,"num_write_kb":3720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":87,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.140040+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:50:35.140040+0000","last_peered":"2024-04-23T23:50:35.140040+0000","last_clean":"2024-04-23T23:50:35.140040+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:50:35.140040+0000","last_undegraded":"2024-04-23T23:50:35.140040+0000","last_fullsized":"2024-04-23T23:50:35.140040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"58'301","reported_seq":1943,"reported_epoch":59,"state":"peering","last_fresh":"2024-04-23T23:50:35.140521+0000","last_change":"2024-04-23T23:50:35.140396+0000","last_active":"2024-04-23T23:50:35.140396+0000","last_peered":"2024-04-23T23:50:35.022465+0000","last_clean":"2024-04-23T23:50:09.421269+0000","last_became_active":"2024-04-23T23:50:34.090479+0000","last_became_peered":"2024-04-23T23:50:34.090479+0000","last_unstale":"2024-04-23T23:50:35.140521+0000","last_undegraded":"2024-04-23T23:50:35.140521+0000","last_fullsized":"2024-04-23T23:50:35.140521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010602859999999999,"stat_sum":{"num_bytes":15803387,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":3096,"num_read_kb":7792,"num_write":304,"num_write_kb":12985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.946291+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:50:33.946291+0000","last_peered":"2024-04-23T23:50:33.946291+0000","last_clean":"2024-04-23T23:50:33.946291+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:50:33.946291+0000","last_undegraded":"2024-04-23T23:50:33.946291+0000","last_fullsized":"2024-04-23T23:50:33.946291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"59'160","reported_seq":898,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.139914+0000","last_change":"2024-04-23T23:49:53.327533+0000","last_active":"2024-04-23T23:50:35.139914+0000","last_peered":"2024-04-23T23:50:35.139914+0000","last_clean":"2024-04-23T23:50:35.139914+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:50:35.139914+0000","last_undegraded":"2024-04-23T23:50:35.139914+0000","last_fullsized":"2024-04-23T23:50:35.139914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:19.576833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056687800000000004,"stat_sum":{"num_bytes":47350539,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":1505,"num_read_kb":3184,"num_write":137,"num_write_kb":1586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":87,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.139973+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:50:35.139973+0000","last_peered":"2024-04-23T23:50:35.139973+0000","last_clean":"2024-04-23T23:50:35.139973+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:50:35.139973+0000","last_undegraded":"2024-04-23T23:50:35.139973+0000","last_fullsized":"2024-04-23T23:50:35.139973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"58'299","reported_seq":4590,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-23T23:50:34.089473+0000","last_change":"2024-04-23T23:50:34.089473+0000","last_active":"2024-04-23T23:50:34.089473+0000","last_peered":"2024-04-23T23:50:34.089473+0000","last_clean":"2024-04-23T23:50:08.486936+0000","last_became_active":"2024-04-23T23:50:34.089119+0000","last_became_peered":"2024-04-23T23:50:34.089119+0000","last_unstale":"2024-04-23T23:50:34.089473+0000","last_undegraded":"2024-04-23T23:50:33.953696+0000","last_fullsized":"2024-04-23T23:50:33.953022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154289,"stat_sum":{"num_bytes":37174179,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":4879,"num_read_kb":21571,"num_write":362,"num_write_kb":8929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":8}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'163","reported_seq":1139,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.295692+0000","last_change":"2024-04-23T23:49:53.322781+0000","last_active":"2024-04-23T23:50:34.295692+0000","last_peered":"2024-04-23T23:50:34.295692+0000","last_clean":"2024-04-23T23:50:34.295692+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:50:34.295692+0000","last_undegraded":"2024-04-23T23:50:34.295692+0000","last_fullsized":"2024-04-23T23:50:34.295692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:30.506102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054288899999999996,"stat_sum":{"num_bytes":7611547,"num_objects":31,"num_object_clones":12,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":1715,"num_read_kb":4510,"num_write":152,"num_write_kb":1213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.952847+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:50:33.952847+0000","last_peered":"2024-04-23T23:50:33.952847+0000","last_clean":"2024-04-23T23:50:33.952847+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:50:33.952847+0000","last_undegraded":"2024-04-23T23:50:33.952847+0000","last_fullsized":"2024-04-23T23:50:33.952847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:39.497845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"58'182","reported_seq":837,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.139724+0000","last_change":"2024-04-23T23:49:53.327404+0000","last_active":"2024-04-23T23:50:35.139724+0000","last_peered":"2024-04-23T23:50:35.139724+0000","last_clean":"2024-04-23T23:50:35.139724+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:50:35.139724+0000","last_undegraded":"2024-04-23T23:50:35.139724+0000","last_fullsized":"2024-04-23T23:50:35.139724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:04.791199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043824499999999998,"stat_sum":{"num_bytes":18631584,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":872,"num_read_kb":3913,"num_write":156,"num_write_kb":4021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T23:50:32.953680+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:50:32.953680+0000","last_peered":"2024-04-23T23:50:32.953680+0000","last_clean":"2024-04-23T23:50:32.953680+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:50:32.953680+0000","last_undegraded":"2024-04-23T23:50:32.953680+0000","last_fullsized":"2024-04-23T23:50:32.953680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"58'204","reported_seq":1087,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.136044+0000","last_change":"2024-04-23T23:49:53.322700+0000","last_active":"2024-04-23T23:50:35.136044+0000","last_peered":"2024-04-23T23:50:35.136044+0000","last_clean":"2024-04-23T23:50:35.136044+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:50:35.136044+0000","last_undegraded":"2024-04-23T23:50:35.136044+0000","last_fullsized":"2024-04-23T23:50:35.136044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:51.554527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085923399999999995,"stat_sum":{"num_bytes":22713518,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":1368,"num_read_kb":3460,"num_write":162,"num_write_kb":1555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.946017+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:50:33.946017+0000","last_peered":"2024-04-23T23:50:33.946017+0000","last_clean":"2024-04-23T23:50:33.946017+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:50:33.946017+0000","last_undegraded":"2024-04-23T23:50:33.946017+0000","last_fullsized":"2024-04-23T23:50:33.946017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:28:10.960350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"59'229","reported_seq":1145,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.138253+0000","last_change":"2024-04-23T23:50:33.055605+0000","last_active":"2024-04-23T23:50:35.138253+0000","last_peered":"2024-04-23T23:50:35.138253+0000","last_clean":"2024-04-23T23:50:35.138253+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:50:35.138253+0000","last_undegraded":"2024-04-23T23:50:35.138253+0000","last_fullsized":"2024-04-23T23:50:35.138253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:19:29.257499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011802519999999999,"stat_sum":{"num_bytes":21894485,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":23,"num_read":4222,"num_read_kb":21021,"num_write":249,"num_write_kb":13858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.952856+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:50:33.952856+0000","last_peered":"2024-04-23T23:50:33.952856+0000","last_clean":"2024-04-23T23:50:33.952856+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:50:33.952856+0000","last_undegraded":"2024-04-23T23:50:33.952856+0000","last_fullsized":"2024-04-23T23:50:33.952856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:33.946073+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:50:33.946073+0000","last_peered":"2024-04-23T23:50:33.946073+0000","last_clean":"2024-04-23T23:50:33.946073+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:50:33.946073+0000","last_undegraded":"2024-04-23T23:50:33.946073+0000","last_fullsized":"2024-04-23T23:50:33.946073+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"59'119","reported_seq":353,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.140453+0000","last_change":"2024-04-23T23:50:33.132092+0000","last_active":"2024-04-23T23:50:35.140453+0000","last_peered":"2024-04-23T23:50:35.140453+0000","last_clean":"2024-04-23T23:50:35.140453+0000","last_became_active":"2024-04-23T23:50:32.955054+0000","last_became_peered":"2024-04-23T23:50:32.955054+0000","last_unstale":"2024-04-23T23:50:35.140453+0000","last_undegraded":"2024-04-23T23:50:35.140453+0000","last_fullsized":"2024-04-23T23:50:35.140453+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:58:47.032840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041896000000000001,"stat_sum":{"num_bytes":18248056,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":2414,"num_read_kb":5640,"num_write":103,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":69682,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"58'306","reported_seq":1688,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.387453+0000","last_change":"2024-04-23T23:49:53.322582+0000","last_active":"2024-04-23T23:50:34.387453+0000","last_peered":"2024-04-23T23:50:34.387453+0000","last_clean":"2024-04-23T23:50:34.387453+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:50:34.387453+0000","last_undegraded":"2024-04-23T23:50:34.387453+0000","last_fullsized":"2024-04-23T23:50:34.387453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:13:34.536299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045422700000000003,"stat_sum":{"num_bytes":10454680,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":3337,"num_read_kb":4358,"num_write":284,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"58'190","reported_seq":1476,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.366830+0000","last_change":"2024-04-23T23:49:53.494617+0000","last_active":"2024-04-23T23:50:34.366830+0000","last_peered":"2024-04-23T23:50:34.366830+0000","last_clean":"2024-04-23T23:50:34.366830+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:50:34.366830+0000","last_undegraded":"2024-04-23T23:50:34.366830+0000","last_fullsized":"2024-04-23T23:50:34.366830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:23.724554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039630999999999998,"stat_sum":{"num_bytes":49440238,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":1279,"num_read_kb":6464,"num_write":188,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"58'276","reported_seq":732,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.138210+0000","last_change":"2024-04-23T23:50:33.091564+0000","last_active":"2024-04-23T23:50:35.138210+0000","last_peered":"2024-04-23T23:50:35.138210+0000","last_clean":"2024-04-23T23:50:35.138210+0000","last_became_active":"2024-04-23T23:50:32.954861+0000","last_became_peered":"2024-04-23T23:50:32.954861+0000","last_unstale":"2024-04-23T23:50:35.138210+0000","last_undegraded":"2024-04-23T23:50:35.138210+0000","last_fullsized":"2024-04-23T23:50:35.138210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:46:50.797394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001287243,"stat_sum":{"num_bytes":17544465,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":1411,"num_read_kb":8148,"num_write":321,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":3287017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"58'267","reported_seq":2925,"reported_epoch":58,"state":"active+undersized+remapped","last_fresh":"2024-04-23T23:50:34.373163+0000","last_change":"2024-04-23T23:50:34.146945+0000","last_active":"2024-04-23T23:50:34.373163+0000","last_peered":"2024-04-23T23:50:34.373163+0000","last_clean":"2024-04-23T23:50:08.500524+0000","last_became_active":"2024-04-23T23:50:34.088072+0000","last_became_peered":"2024-04-23T23:50:34.088072+0000","last_unstale":"2024-04-23T23:50:34.373163+0000","last_undegraded":"2024-04-23T23:50:34.373163+0000","last_fullsized":"2024-04-23T23:50:33.953018+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012607440000000001,"stat_sum":{"num_bytes":35679819,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":3210,"num_read_kb":18025,"num_write":266,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"5","objects":42}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"58'170","reported_seq":2372,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.328172+0000","last_change":"2024-04-23T23:49:53.781509+0000","last_active":"2024-04-23T23:50:34.328172+0000","last_peered":"2024-04-23T23:50:34.328172+0000","last_clean":"2024-04-23T23:50:34.328172+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:50:34.328172+0000","last_undegraded":"2024-04-23T23:50:34.328172+0000","last_fullsized":"2024-04-23T23:50:34.328172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:39:09.055583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090598199999999995,"stat_sum":{"num_bytes":49676575,"num_objects":34,"num_object_clones":11,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":2739,"num_read_kb":20341,"num_write":174,"num_write_kb":9589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"58'257","reported_seq":1348,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.237072+0000","last_change":"2024-04-23T23:49:53.781245+0000","last_active":"2024-04-23T23:50:34.237072+0000","last_peered":"2024-04-23T23:50:34.237072+0000","last_clean":"2024-04-23T23:50:34.237072+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:50:34.237072+0000","last_undegraded":"2024-04-23T23:50:34.237072+0000","last_fullsized":"2024-04-23T23:50:34.237072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:01.334952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071364900000000001,"stat_sum":{"num_bytes":13152069,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":2109,"num_read_kb":9517,"num_write":351,"num_write_kb":748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"58'360","reported_seq":1679,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.324339+0000","last_change":"2024-04-23T23:49:53.781604+0000","last_active":"2024-04-23T23:50:34.324339+0000","last_peered":"2024-04-23T23:50:34.324339+0000","last_clean":"2024-04-23T23:50:34.324339+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:50:34.324339+0000","last_undegraded":"2024-04-23T23:50:34.324339+0000","last_fullsized":"2024-04-23T23:50:34.324339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:56:26.913015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097850600000000008,"stat_sum":{"num_bytes":44184650,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2418,"num_read_kb":13726,"num_write":488,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"58'121","reported_seq":3450,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.370463+0000","last_change":"2024-04-23T23:49:53.494621+0000","last_active":"2024-04-23T23:50:34.370463+0000","last_peered":"2024-04-23T23:50:34.370463+0000","last_clean":"2024-04-23T23:50:34.370463+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:50:34.370463+0000","last_undegraded":"2024-04-23T23:50:34.370463+0000","last_fullsized":"2024-04-23T23:50:34.370463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:43:11.923753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400901,"stat_sum":{"num_bytes":16954708,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":3289,"num_read_kb":7642,"num_write":88,"num_write_kb":1518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"58'109","reported_seq":711,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.317957+0000","last_change":"2024-04-23T23:49:53.780895+0000","last_active":"2024-04-23T23:50:34.317957+0000","last_peered":"2024-04-23T23:50:34.317957+0000","last_clean":"2024-04-23T23:50:34.317957+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:50:34.317957+0000","last_undegraded":"2024-04-23T23:50:34.317957+0000","last_fullsized":"2024-04-23T23:50:34.317957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:24.555689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049304299999999998,"stat_sum":{"num_bytes":6144361,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":543,"num_read_kb":2664,"num_write":103,"num_write_kb":2148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"58'305","reported_seq":1448,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-23T23:50:35.136266+0000","last_change":"2024-04-23T23:49:53.323023+0000","last_active":"2024-04-23T23:50:35.136266+0000","last_peered":"2024-04-23T23:50:35.136266+0000","last_clean":"2024-04-23T23:50:35.136266+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:50:35.136266+0000","last_undegraded":"2024-04-23T23:50:35.136266+0000","last_fullsized":"2024-04-23T23:50:35.136266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:40:50.709020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000902432,"stat_sum":{"num_bytes":56651211,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":1705,"num_read_kb":12995,"num_write":316,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"58'300","reported_seq":2085,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T23:50:34.302033+0000","last_change":"2024-04-23T23:49:53.780967+0000","last_active":"2024-04-23T23:50:34.302033+0000","last_peered":"2024-04-23T23:50:34.302033+0000","last_clean":"2024-04-23T23:50:34.302033+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:50:34.302033+0000","last_undegraded":"2024-04-23T23:50:34.302033+0000","last_fullsized":"2024-04-23T23:50:34.302033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:33.641424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053415600000000004,"stat_sum":{"num_bytes":40828290,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":2948,"num_read_kb":12488,"num_write":279,"num_write_kb":14658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":907717996,"num_objects":1184,"num_object_clones":291,"num_object_copies":2368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1184,"num_whiteouts":480,"num_read":82469,"num_read_kb":323791,"num_write":8867,"num_write_kb":187681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":12091085,"num_keys_recovered":59,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201236480,"data_stored":261477756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":7933,"ondisk_log_size":7933,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168578,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69264,"kb_used_data":40708,"kb_used_omap":10,"kb_used_meta":28533,"kb_avail":94302576,"statfs":{"total":96636764160,"available":96565837824,"internally_reserved":0,"allocated":41684992,"data_stored":52030559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":29218058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42724,"kb_used_data":7388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329116,"statfs":{"total":96636764160,"available":96593014784,"internally_reserved":0,"allocated":7565312,"data_stored":8978211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64552,"kb_used_data":25052,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307288,"statfs":{"total":96636764160,"available":96570662912,"internally_reserved":0,"allocated":25653248,"data_stored":31939686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77752,"kb_used_data":42320,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94294088,"statfs":{"total":96636764160,"available":96557146112,"internally_reserved":0,"allocated":43335680,"data_stored":56999271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34264,"kb_used_data":3076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337576,"statfs":{"total":96636764160,"available":96601677824,"internally_reserved":0,"allocated":3149824,"data_stored":3686998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86744,"kb_used_data":51400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94285096,"statfs":{"total":96636764160,"available":96547938304,"internally_reserved":0,"allocated":52633600,"data_stored":69821432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":20296,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":20783104,"data_stored":27209776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35632,"kb_used_data":4452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336208,"statfs":{"total":96636764160,"available":96600276992,"internally_reserved":0,"allocated":4558848,"data_stored":6041174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24383488,"data_stored":31046717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6918144,"data_stored":8695105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3907584,"data_stored":5750475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41009152,"data_stored":51727958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22278144,"data_stored":29049458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53878784,"data_stored":71433506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3579904,"data_stored":4477643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45281280,"data_stored":59296894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:50:37.608 INFO:tasks.thrashosds.thrasher:Setting 3.1c to [6, 3, 4, 1] 2024-04-23T23:50:37.608 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1c', '6', '3', '4', '1'] 2024-04-23T23:50:37.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1c 6 3 4 1 2024-04-23T23:50:37.615 INFO:teuthology.orchestra.run.smithi092.stdout:3839 read 0xd7c70ce thru 0xd7cab6e (0x3aa1 bytes) 2024-04-23T23:50:37.615 INFO:teuthology.orchestra.run.smithi092.stdout:3841 writesame 0x7d9a069 thru 0x7da5d7c data_size 0x2(0xbd14 bytes) 2024-04-23T23:50:37.636 INFO:teuthology.orchestra.run.smithi092.stdout:3842 trunc from 0xdd63e00 to 0x8e11ce9 2024-04-23T23:50:37.636 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:37.650+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:37.645 INFO:teuthology.orchestra.run.smithi092.stdout:3844 write 0x3381e42 thru 0x338fc57 (0xde16 bytes) 2024-04-23T23:50:37.647 INFO:teuthology.orchestra.run.smithi092.stdout:3845 punch from 0x821e45c to 0x82231de, (0x4d82 bytes) 2024-04-23T23:50:37.647 INFO:teuthology.orchestra.run.smithi092.stdout:3848 punch from 0x6fbe4e5 to 0x6fc8490, (0x9fab bytes) 2024-04-23T23:50:37.647 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.647 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:37.647 INFO:teuthology.orchestra.run.smithi092.stdout:3854 write 0x3c6b4d4 thru 0x3c70f1b (0x5a48 bytes) 2024-04-23T23:50:37.654 INFO:teuthology.orchestra.run.smithi092.stdout:3855 trunc from 0x8e11ce9 to 0x3efe536 2024-04-23T23:50:37.672 INFO:teuthology.orchestra.run.smithi092.stdout:3856 trunc from 0x3efe536 to 0xbf87c2d 2024-04-23T23:50:37.678 INFO:teuthology.orchestra.run.smithi092.stdout:3857 punch from 0x5c6e07f to 0x5c7cd93, (0xed14 bytes) 2024-04-23T23:50:37.678 INFO:teuthology.orchestra.run.smithi092.stdout:3860 clone 27 order 25 su 262144 sc 8 2024-04-23T23:50:37.698 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.698 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.707 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:37.751 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:37.765+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:37.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:37.809 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.809 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:37.848 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:37.862+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:37.924 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:37.928 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:37.942+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:37.941 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:37.942 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:37.958 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:37.961 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:37.961 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:37.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:37.974 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:37.992 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:38.009 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:38.009 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:38.026 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:38.026 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:38.029 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:38.043+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:38.043 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:38.044 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:38.075 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:38.076 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:38.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:38.130 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:38.144+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:38.138 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:50:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:set 3.1c pg_upmap_items mapping to [6->3,4->1] 2024-04-23T23:50:38.231 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:38.245+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:38.232 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:38.233 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:38.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:38.332 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:38.346+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:38.344 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.344 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:38.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:38.344 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.242751578, 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.345 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:50:38.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.240157247, 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:38.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:38.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:50:38.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1285018080000002, 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:50:38.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1284833680000004, 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.360 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:50:38.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:50:38.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:50:38.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1284642619999996, 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1284457530000003, 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1283427039999996, 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:50:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1283230849999999, 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:50:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1282176100000001, 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:50:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1281924249999999, 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.12815888, 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:50:38.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:50:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1281351170000002, 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:50:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1281140760000001, 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:50:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:50:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1280458409999996, 2024-04-23T23:50:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:50:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1280213730000002, 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1279956750000002, 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1279739089999996, 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:50:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:50:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.127953443, 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:50:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:50:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1278755839999999, 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:50:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:50:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:50:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1278459329999997, 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:50:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.434 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:38.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:38.436 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:38.449+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:38.536 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:38.550+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:38.538 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:38.538 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:38.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:38.637 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:38.651+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:38.649 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:38.650 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:38.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:38.738 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:38.751+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.958886124999999, 2024-04-23T23:50:38.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:38.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:50:38.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.956842141999999, 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:50:38.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:50:38.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:50:38.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:38.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.954864894, 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.954834261, 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:38.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:50:38.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.952891411, 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:50:38.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.950806202000001, 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.948796771, 2024-04-23T23:50:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:38.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:50:38.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.946896803, 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:50:38.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.942848274999999, 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:38.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.940803599000001, 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.938730026, 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:38.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:50:38.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.93684187, 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:50:38.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:50:38.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.936821093000001, 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:50:38.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:50:38.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.928829599, 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:50:38.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.926894617, 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.926868521999999, 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:38.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5451510580000001, 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:50:38.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:50:38.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5451086390000004, 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5450726819999998, 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:38.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5450483239999997, 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:50:38.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:50:38.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:38.849 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:38.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:38.851 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:38.864+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:38.951 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:38.965+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:38.995 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:38.995 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:39.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:39.052 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:39.066+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:39.106 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:39.106 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:39.106 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:39.106 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:39.106 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:39.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:39.160 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:39.174+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.721524978, 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.721058991, 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.720552442000001, 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.720256534000001, 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:50:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:50:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.720069520999999, 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:50:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:50:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.719867851, 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:50:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.719727589, 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.719481854, 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:50:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:50:39.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.719204329, 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:50:39.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:50:39.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.718961757000001, 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.718783393000001, 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.718500801999999, 2024-04-23T23:50:39.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:50:39.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.718308318, 2024-04-23T23:50:39.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.717880377, 2024-04-23T23:50:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:50:39.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.717728043999999, 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:39.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:50:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.717624961, 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.717308015, 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.288 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.717066588, 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.716842041, 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:39.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:39.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.716587144, 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.304 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:39.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:39.306 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:39.319+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:39.407 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:39.421+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:39.449 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:39.449 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:39.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:39.508 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:39.522+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:39.561 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:39.561 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:39.561 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:39.561 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:39.561 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:39.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:39.611 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:39.625+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:39.711 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:39.725+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:39.715 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.269271721999999, 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:39.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:39.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.521612761999997, 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:39.720 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.679743457000001, 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:39.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:39.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:39.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:39.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:39.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:39.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:39.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:39.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.732 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:58072 3.9 e54/22 3:967d967a:::rbd_data.117966a158ce.0000000000000002:head v 54'280, mlcod=53'276)", 2024-04-23T23:50:39.732 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:39.732 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5353865950000003, 2024-04-23T23:50:39.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003273686, 2024-04-23T23:50:39.732 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.733 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:39.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:39.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193448+0000", 2024-04-23T23:50:39.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2024-04-23T23:50:39.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193458+0000", 2024-04-23T23:50:39.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0628000000000001e-05 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193469+0000", 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380000000000001e-05 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193479+0000", 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193529+0000", 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-05 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193551+0000", 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196689+0000", 2024-04-23T23:50:39.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031378420000000001 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196721+0000", 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2400000000000001e-05 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4597652270000001, 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:50:39.743 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:50:39.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:50:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4099703400000001, 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:39.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:50:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4088395880000002, 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:50:39.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63551 3.9 e58/22 3:94220160:::rbd_data.11798ae5b3f1.000000000000004c:head v 58'303, mlcod=58'301)", 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.3704961640000004, 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029639520000000002, 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6400000000000001e-07 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358354+0000", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5396e-05 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358366+0000", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2289e-05 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358373+0000", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4150000000000004e-06 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358441+0000", 2024-04-23T23:50:39.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8459000000000003e-05 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358462+0000", 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1078e-05 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.361257+0000", 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002794745 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.361302+0000", 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4805999999999998e-05 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.3633452720000001, 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:50:39.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:50:39.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:50:39.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:50:39.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:50:39.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:50:39.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:50:39.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:50:39.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:50:39.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.769 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.770 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:50:39.770 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:39.770 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.3605748980000003, 2024-04-23T23:50:39.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:50:39.770 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.770 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.771 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:39.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:39.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:50:39.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:39.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:50:39.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:39.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:50:39.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:50:39.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:50:39.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:50:39.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:50:39.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:50:39.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:50:39.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:50:39.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:50:39.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63748 3.4 e58/22 3:21ed4642:::rbd_data.11798ae5b3f1.0000000000000064:head v 58'132, mlcod=58'128)", 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.3321838770000003, 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003193577, 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396651+0000", 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396657+0000", 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8980000000000001e-06 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396667+0000", 2024-04-23T23:50:39.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0288e-05 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396673+0000", 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396718+0000", 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396736+0000", 2024-04-23T23:50:39.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8071e-05 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399806+0000", 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003069881 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399844+0000", 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8056999999999998e-05 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64075 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'141, mlcod=58'139)", 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2658730059999996, 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028754750000000002, 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462969+0000", 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9519999999999994e-06 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462982+0000", 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3095000000000001e-05 2024-04-23T23:50:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462992+0000", 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8719999999999997e-06 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462999+0000", 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463049+0000", 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0837999999999997e-05 2024-04-23T23:50:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463067+0000", 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7374e-05 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.465799+0000", 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027323080000000001 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.465836+0000", 2024-04-23T23:50:39.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7314999999999998e-05 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2651890689999998, 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:50:39.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2627709869999997, 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:50:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64292 3.9 e58/22 3:957b2643:::rbd_data.11798ae5b3f1.000000000000011d:head v 58'325, mlcod=58'323)", 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1992934689999997, 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003216596, 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529541+0000", 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529546+0000", 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6619999999999996e-06 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529557+0000", 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1035e-05 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529564+0000", 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0219999999999999e-06 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529615+0000", 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0899000000000002e-05 2024-04-23T23:50:39.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529636+0000", 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1739999999999999e-05 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532716+0000", 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003079755 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532757+0000", 2024-04-23T23:50:39.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64347 3.4 e58/22 3:23ee6249:::rbd_data.11798ae5b3f1.000000000000013a:head v 58'151, mlcod=58'149)", 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1804808759999998, 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036492059999999999, 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548354+0000", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6899999999999996e-07 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548359+0000", 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548372+0000", 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3050999999999999e-05 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548379+0000", 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548448+0000", 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9326999999999995e-05 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548466+0000", 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659000000000002e-05 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551946+0000", 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034799850000000001 2024-04-23T23:50:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552002+0000", 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6508999999999997e-05 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64341 3.0 e58/22 3:0635a7b0:::rbd_data.11798ae5b3f1.0000000000000138:head v 58'169, mlcod=58'166)", 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1800729729999997, 2024-04-23T23:50:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032676770000000001, 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548766+0000", 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548777+0000", 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1453e-05 2024-04-23T23:50:39.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548784+0000", 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6460000000000004e-06 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548833+0000", 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8498000000000002e-05 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548852+0000", 2024-04-23T23:50:39.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9142000000000001e-05 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551992+0000", 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031404649999999998 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552029+0000", 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6381999999999998e-05 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 58/22 rep_tid 964 entries 58'170 (0'0) error 3:0535656d:::rbd_data.11798ae5b3f1.000000000000013d:head by client.4473.0:64343 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 58'166)", 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.179751886, 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029602270000000002, 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8800000000000003e-07 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549090+0000", 2024-04-23T23:50:39.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3470000000000001e-06 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549106+0000", 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6222000000000002e-05 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549120+0000", 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4260000000000001e-05 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549190+0000", 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0184000000000004e-05 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552042+0000", 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028517260000000002 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1724786050000002, 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:50:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:67439 3.0 e58/22 3:0718c4b2:::rbd_data.117911508d31.0000000000000006:head v 58'177, mlcod=58'175)", 2024-04-23T23:50:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8197100419999996, 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036877949999999998, 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909125+0000", 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909535+0000", 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040998899999999998 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909550+0000", 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5092000000000001e-05 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909558+0000", 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6790000000000005e-06 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909610+0000", 2024-04-23T23:50:39.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1996000000000002e-05 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909638+0000", 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8231000000000001e-05 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912774+0000", 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031359740000000001 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912812+0000", 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7710999999999999e-05 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:39.825 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:39.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:39.827 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:39.840+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:39.927 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:39.941+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:39.941 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:39.941 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:39.942 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:39.942 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:39.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:40.029 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:40.043+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:40.052 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:40.053 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:40.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:40.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:40.129 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:40.143+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:40.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:50:40.179 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:40.180 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:40.197 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:40.197 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:40.208 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.790747924, 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:50:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:50:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.790738043999999, 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:50:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.790527697, 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.788696357999999, 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:50:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:50:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.786626727, 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:50:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:50:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:50:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274536490999999, 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:50:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:50:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274510834999999, 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:50:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274443345, 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:50:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:50:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274398284, 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:50:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274361784, 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:50:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:50:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274344825, 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:50:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274266141, 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:50:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274199107999999, 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:50:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:50:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.274065507, 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:50:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.273646827, 2024-04-23T23:50:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:50:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:50:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.273496185000001, 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:50:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:50:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.273370803000001, 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:50:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:50:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.273155286, 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:50:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.272917952, 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:50:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.272678057, 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:50:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:40.305 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:40.306 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:40.307 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:40.307 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:40.308 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:40.308 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:40.309 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:40.309 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:40.309 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:40.310 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:40.310 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:40.320+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:40.317 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:40.339 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:40.407 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:40.421+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:40.413 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:40.413 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:40.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:40.508 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:40.522+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:40.563 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:40.563 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:40.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:40.609 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:40.624+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:40.710 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone26 from 0xbf87c2d (overlap 0x3efe536) to 0x25cbcba 2024-04-23T23:50:40.711 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:40.724+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:40.726 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:40.727 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:40.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:40.819 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:40.833+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:40.820 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-04-23T23:50:40.826 INFO:teuthology.orchestra.run.smithi092.stdout:3861 read 0xb7246d3 thru 0xb72bf22 (0x7850 bytes) 2024-04-23T23:50:40.827 INFO:teuthology.orchestra.run.smithi092.stdout:3863 write 0x9fb4bd9 thru 0x9fb558e (0x9b6 bytes) 2024-04-23T23:50:40.852 INFO:teuthology.orchestra.run.smithi092.stdout:3864 trunc from 0xbf87c2d to 0x3ceaec5 2024-04-23T23:50:40.897 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:40.897 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:40.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:40.909 INFO:teuthology.orchestra.run.smithi092.stdout:3866 punch from 0xc9008 to 0xc9d8d, (0xd85 bytes) 2024-04-23T23:50:40.909 INFO:teuthology.orchestra.run.smithi092.stdout:3867 read 0x3540d79 thru 0x3550b08 (0xfd90 bytes) 2024-04-23T23:50:40.910 INFO:teuthology.orchestra.run.smithi092.stdout:3868 read 0x341212a thru 0x341709d (0x4f74 bytes) 2024-04-23T23:50:40.910 INFO:teuthology.orchestra.run.smithi092.stdout:3869 read 0x2234fcc thru 0x2240947 (0xb97c bytes) 2024-04-23T23:50:40.924 INFO:teuthology.orchestra.run.smithi092.stdout:3870 write 0xadba8da thru 0xadc1488 (0x6baf bytes) 2024-04-23T23:50:40.926 INFO:teuthology.orchestra.run.smithi092.stdout:3872 write 0x56f591c thru 0x56f6607 (0xcec bytes) 2024-04-23T23:50:40.933 INFO:teuthology.orchestra.run.smithi092.stdout:3873 write 0xbebfb64 thru 0xbece8c0 (0xed5d bytes) 2024-04-23T23:50:40.935 INFO:teuthology.orchestra.run.smithi092.stdout:3874 writesame 0x314f799 thru 0x3158cb1 data_size 0x3(0x9519 bytes) 2024-04-23T23:50:40.943 INFO:teuthology.orchestra.run.smithi092.stdout:3875 write 0x7a5c430 thru 0x7a65a3d (0x960e bytes) 2024-04-23T23:50:40.944 INFO:teuthology.orchestra.run.smithi092.stdout:3876 read 0xad8291f thru 0xad82933 (0x15 bytes) 2024-04-23T23:50:40.945 INFO:teuthology.orchestra.run.smithi092.stdout:3877 write 0x4aca3bf thru 0x4acf675 (0x52b7 bytes) 2024-04-23T23:50:40.947 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:40.948 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:40.962+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:40.951 INFO:teuthology.orchestra.run.smithi092.stdout:3880 write 0xc130dcf thru 0xc13a148 (0x937a bytes) 2024-04-23T23:50:40.953 INFO:teuthology.orchestra.run.smithi092.stdout:3881 punch from 0x806270 to 0x8098e4, (0x3674 bytes) 2024-04-23T23:50:40.960 INFO:teuthology.orchestra.run.smithi092.stdout:3882 write 0xdf7ed92 thru 0xdf87516 (0x8785 bytes) 2024-04-23T23:50:40.962 INFO:teuthology.orchestra.run.smithi092.stdout:3885 writesame 0x208ef74 thru 0x209e114 data_size 0x1d(0xf1a1 bytes) 2024-04-23T23:50:40.965 INFO:teuthology.orchestra.run.smithi092.stdout:3886 trunc from 0xdf87517 to 0xde7490b 2024-04-23T23:50:40.971 INFO:teuthology.orchestra.run.smithi092.stdout:3887 writesame 0xc9a27de thru 0xc9b165a data_size 0x2f(0xee7d bytes) 2024-04-23T23:50:40.974 INFO:teuthology.orchestra.run.smithi092.stdout:3888 writesame 0x2cd41df thru 0x2cd9cd5 data_size 0x1(0x5af7 bytes) 2024-04-23T23:50:40.988 INFO:teuthology.orchestra.run.smithi092.stdout:3889 writesame 0xca5628a thru 0xca61323 data_size 0x6e(0xb09a bytes) 2024-04-23T23:50:40.989 INFO:teuthology.orchestra.run.smithi092.stdout:3890 write 0x126f43d thru 0x1270afe (0x16c2 bytes) 2024-04-23T23:50:40.991 INFO:teuthology.orchestra.run.smithi092.stdout:3892 read 0x4034034 thru 0x4039e39 (0x5e06 bytes) 2024-04-23T23:50:40.991 INFO:teuthology.orchestra.run.smithi092.stdout:3893 read 0xe30b28 thru 0xe39f5b (0x9434 bytes) 2024-04-23T23:50:40.992 INFO:teuthology.orchestra.run.smithi092.stdout:3894 write 0xd2b6854 thru 0xd2c5773 (0xef20 bytes) 2024-04-23T23:50:40.994 INFO:teuthology.orchestra.run.smithi092.stdout:3895 read 0x48e4eff thru 0x48e9130 (0x4232 bytes) 2024-04-23T23:50:40.994 INFO:teuthology.orchestra.run.smithi092.stdout:3896 writesame 0x91ca2c4 thru 0x91cac7c data_size 0x1(0x9b9 bytes) 2024-04-23T23:50:40.997 INFO:teuthology.orchestra.run.smithi092.stdout:3897 punch from 0x5d5aec1 to 0x5d63260, (0x839f bytes) 2024-04-23T23:50:41.001 INFO:teuthology.orchestra.run.smithi092.stdout:3899 trunc from 0xde7490b to 0xa5a92b0 2024-04-23T23:50:41.013 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:41.027+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:41.035 INFO:teuthology.orchestra.run.smithi092.stdout:3900 read 0x23f3d15 thru 0x24019bc (0xdca8 bytes) 2024-04-23T23:50:41.035 INFO:teuthology.orchestra.run.smithi092.stdout:3901 punch from 0x69a37a4 to 0x69ac094, (0x88f0 bytes) 2024-04-23T23:50:41.046 INFO:teuthology.orchestra.run.smithi092.stdout:3902 trunc from 0xa5a92b0 to 0x1cf3832 2024-04-23T23:50:41.053 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:41.053 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:41.055 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.055 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:41.068 INFO:teuthology.orchestra.run.smithi092.stdout:3905 writesame 0x8cac229 thru 0x8cae7a9 data_size 0x1(0x2581 bytes) 2024-04-23T23:50:41.069 INFO:teuthology.orchestra.run.smithi092.stdout:3906 read 0x7322be7 thru 0x732ff0e (0xd328 bytes) 2024-04-23T23:50:41.077 INFO:teuthology.orchestra.run.smithi092.stdout:3907 write 0xa4ab12d thru 0xa4b4656 (0x952a bytes) 2024-04-23T23:50:41.088 INFO:teuthology.orchestra.run.smithi092.stdout:3908 writesame 0xcfe3f1e thru 0xcfee6c6 data_size 0x39(0xa7a9 bytes) 2024-04-23T23:50:41.090 INFO:teuthology.orchestra.run.smithi092.stdout:3910 write 0x579332c thru 0x579418d (0xe62 bytes) 2024-04-23T23:50:41.091 INFO:teuthology.orchestra.run.smithi092.stdout:3911 writesame 0xc5a6305 thru 0xc5a6e1c data_size 0x14(0xb18 bytes) 2024-04-23T23:50:41.093 INFO:teuthology.orchestra.run.smithi092.stdout:3913 read 0x5168eb7 thru 0x5171f72 (0x90bc bytes) 2024-04-23T23:50:41.101 INFO:teuthology.orchestra.run.smithi092.stdout:3914 trunc from 0xcfee6c7 to 0x62d6648 2024-04-23T23:50:41.110 INFO:teuthology.orchestra.run.smithi092.stdout:3915 read 0x10993ae thru 0x109ec67 (0x58ba bytes) 2024-04-23T23:50:41.110 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:41.111 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:41.111 INFO:teuthology.orchestra.run.smithi092.stdout:3919 write 0x3517774 thru 0x3518848 (0x10d5 bytes) 2024-04-23T23:50:41.114 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:41.128+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:41.128 INFO:teuthology.orchestra.run.smithi092.stdout:3920 write 0xe37d9e4 thru 0xe37f358 (0x1975 bytes) 2024-04-23T23:50:41.129 INFO:teuthology.orchestra.run.smithi092.stdout:3921 punch from 0x29efc98 to 0x29f010b, (0x473 bytes) 2024-04-23T23:50:41.129 INFO:teuthology.orchestra.run.smithi092.stdout:3924 read 0xc72132 thru 0xc7d211 (0xb0e0 bytes) 2024-04-23T23:50:41.129 INFO:teuthology.orchestra.run.smithi092.stdout:3925 read 0xa921ff1 thru 0xa925f9f (0x3faf bytes) 2024-04-23T23:50:41.130 INFO:teuthology.orchestra.run.smithi092.stdout:3927 punch from 0x7d81337 to 0x7d8eb9f, (0xd868 bytes) 2024-04-23T23:50:41.130 INFO:teuthology.orchestra.run.smithi092.stdout:3928 read 0x1ec4367 thru 0x1eccdcb (0x8a65 bytes) 2024-04-23T23:50:41.130 INFO:teuthology.orchestra.run.smithi092.stdout:3929 clone 28 order 22 su 65536 sc 5 2024-04-23T23:50:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:50:41.173 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.173 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:41.227 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:41.241+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:41.284 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.284 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:41.335 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:41.348+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:41.417 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:41.430+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:41.436 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.436 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:41.517 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:41.531+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:41.546 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.546 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:41.618 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:41.632+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:41.658 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.658 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:41.719 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:41.733+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:41.769 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.769 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:41.821 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:41.834+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:41.919 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:41.919 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:41.921 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:41.935+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:41.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:42.022 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:42.036+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:42.030 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:42.030 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:42.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:42.123 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:42.137+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:42.141 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:42.141 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:42.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:42.235 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:42.249+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:42.240 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone27 2024-04-23T23:50:42.300 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:42.300 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:42.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:42.325 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:42.339+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:42.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:42.423 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-04-23T23:50:42.426 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:42.439+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:42.466 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:42.466 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:42.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:42.477 INFO:teuthology.orchestra.run.smithi092.stdout:3930 write 0x635187f thru 0x6359867 (0x7fe9 bytes) 2024-04-23T23:50:42.491 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:42.491 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:42.496 INFO:teuthology.orchestra.run.smithi092.stdout:3932 trunc from 0xe37f359 to 0x5e65276 2024-04-23T23:50:42.517 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:42.517 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:42.521 INFO:teuthology.orchestra.run.smithi092.stdout:3933 read 0x2717fc3 thru 0x2722595 (0xa5d3 bytes) 2024-04-23T23:50:42.521 INFO:teuthology.orchestra.run.smithi092.stdout:3935 write 0x4fc2767 thru 0x4fc6fdf (0x4879 bytes) 2024-04-23T23:50:42.526 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:42.540+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:42.528 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.528 INFO:teuthology.orchestra.run.smithi092.stdout:3938 punch from 0x2e6b6db to 0x2e6ee7d, (0x37a2 bytes) 2024-04-23T23:50:42.528 INFO:teuthology.orchestra.run.smithi092.stdout:3942 read 0x4619f5d thru 0x4623a32 (0x9ad6 bytes) 2024-04-23T23:50:42.528 INFO:teuthology.orchestra.run.smithi092.stdout:3943 punch from 0x1039b1b to 0x1043ef2, (0xa3d7 bytes) 2024-04-23T23:50:42.536 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:42.538 INFO:teuthology.orchestra.run.smithi092.stdout:3944 write 0x9ff1fd1 thru 0x9ff32a1 (0x12d1 bytes) 2024-04-23T23:50:42.541 INFO:teuthology.orchestra.run.smithi092.stdout:3946 read 0x3fd42e6 thru 0x3fd854f (0x426a bytes) 2024-04-23T23:50:42.544 INFO:teuthology.orchestra.run.smithi092.stdout:3948 trunc from 0x9ff32a2 to 0x79dc6fa 2024-04-23T23:50:42.554 INFO:teuthology.orchestra.run.smithi092.stdout:3950 read 0x5c1192b thru 0x5c193ef (0x7ac5 bytes) 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:3953 punch from 0x30fa85d to 0x3109cca, (0xf46d bytes) 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:3954 punch from 0xe1b495 to 0xe1d17c, (0x1ce7 bytes) 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:3957 read 0x71b2ff8 thru 0x71beec4 (0xbecd bytes) 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:3958 read 0x6f3dfdf thru 0x6f3e36f (0x391 bytes) 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:3959 punch from 0x3eee5a5 to 0x3efad0f, (0xc76a bytes) 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.555 INFO:teuthology.orchestra.run.smithi092.stdout:3963 write 0x5f5f645 thru 0x5f6a312 (0xacce bytes) 2024-04-23T23:50:42.556 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:42.556 INFO:teuthology.orchestra.run.smithi092.stdout:3964 writesame 0x628b193 thru 0x6294dcf data_size 0x25(0x9c3d bytes) 2024-04-23T23:50:42.558 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.558 INFO:teuthology.orchestra.run.smithi092.stdout:3968 read 0x6cd965f thru 0x6ce107b (0x7a1d bytes) 2024-04-23T23:50:42.570 INFO:teuthology.orchestra.run.smithi092.stdout:3969 write 0xd9ce4fd thru 0xd9d48ed (0x63f1 bytes) 2024-04-23T23:50:42.574 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.574 INFO:teuthology.orchestra.run.smithi092.stdout:3973 read 0x60728b9 thru 0x60797b5 (0x6efd bytes) 2024-04-23T23:50:42.575 INFO:teuthology.orchestra.run.smithi092.stdout:3975 punch from 0x6ff6a6c to 0x6fff059, (0x85ed bytes) 2024-04-23T23:50:42.575 INFO:teuthology.orchestra.run.smithi092.stdout:3976 read 0x69bd616 thru 0x69caa23 (0xd40e bytes) 2024-04-23T23:50:42.575 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:42.576 INFO:teuthology.orchestra.run.smithi092.stdout:3979 writesame 0x77fde3b thru 0x7808fb2 data_size 0x9(0xb178 bytes) 2024-04-23T23:50:42.580 INFO:teuthology.orchestra.run.smithi092.stdout:3980 writesame 0x37abd09 thru 0x37ad2aa data_size 0x1a(0x15a2 bytes) 2024-04-23T23:50:42.586 INFO:teuthology.orchestra.run.smithi092.stdout:3982 read 0x3eaf5c1 thru 0x3ebe1fb (0xec3b bytes) 2024-04-23T23:50:42.587 INFO:teuthology.orchestra.run.smithi092.stdout:3984 write 0x10afea8 thru 0x10b6308 (0x6461 bytes) 2024-04-23T23:50:42.601 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:42.601 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:42.601 INFO:teuthology.orchestra.run.smithi092.stdout:3985 write 0xc92ac2c thru 0xc92ada2 (0x177 bytes) 2024-04-23T23:50:42.607 INFO:teuthology.orchestra.run.smithi092.stdout:3986 trunc from 0xd9d48ee to 0xa7b9e49 2024-04-23T23:50:42.617 INFO:teuthology.orchestra.run.smithi092.stdout:3987 punch from 0x1660343 to 0x1660c10, (0x8cd bytes) 2024-04-23T23:50:42.617 INFO:teuthology.orchestra.run.smithi092.stdout:3988 punch from 0x25ba4e0 to 0x25c1d3f, (0x785f bytes) 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout:3990 write 0x24c8acc thru 0x24d61be (0xd6f3 bytes) 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:42.618 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:42.622 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:42.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:42.627 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:42.641+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:42.629 INFO:teuthology.orchestra.run.smithi092.stdout:3991 punch from 0x7dcac82 to 0x7dcecc2, (0x4040 bytes) 2024-04-23T23:50:42.629 INFO:teuthology.orchestra.run.smithi092.stdout:3993 write 0xa4f78aa thru 0xa4fbb21 (0x4278 bytes) 2024-04-23T23:50:42.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:42.632 INFO:teuthology.orchestra.run.smithi092.stdout:3994 read 0x6a72e8d thru 0x6a7780a (0x497e bytes) 2024-04-23T23:50:42.640 INFO:teuthology.orchestra.run.smithi092.stdout:3995 write 0xc2e74db thru 0xc2f2799 (0xb2bf bytes) 2024-04-23T23:50:42.642 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:42.643 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:42.644 INFO:teuthology.orchestra.run.smithi092.stdout:3996 read 0xa37bdb6 thru 0xa38227f (0x64ca bytes) 2024-04-23T23:50:42.644 INFO:teuthology.orchestra.run.smithi092.stdout:3998 writesame 0xababa06 thru 0xabb02e4 data_size 0x29(0x48df bytes) 2024-04-23T23:50:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:4000 punch from 0xacc80ac to 0xacce9f9, (0x694d bytes) 2024-04-23T23:50:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:4003 write 0x532d16e thru 0x533bb6c (0xe9ff bytes) 2024-04-23T23:50:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:4004 write 0x882d1f7 thru 0x8835305 (0x810f bytes) 2024-04-23T23:50:42.669 INFO:teuthology.orchestra.run.smithi092.stdout:4005 trunc from 0xc2f279a to 0x3ead02 2024-04-23T23:50:42.708 INFO:teuthology.orchestra.run.smithi092.stdout:4006 trunc from 0x3ead02 to 0x7681287 2024-04-23T23:50:42.715 INFO:teuthology.orchestra.run.smithi092.stdout:4007 read 0x4abe315 thru 0x4ac7670 (0x935c bytes) 2024-04-23T23:50:42.715 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.722 INFO:teuthology.orchestra.run.smithi092.stdout:4009 trunc from 0x7681287 to 0xcd6e931 2024-04-23T23:50:42.728 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:42.742+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:42.740 INFO:teuthology.orchestra.run.smithi092.stdout:4010 trunc from 0xcd6e931 to 0x19f9b76 2024-04-23T23:50:42.745 INFO:teuthology.orchestra.run.smithi092.stdout:4011 read 0x1210970 thru 0x1218f23 (0x85b4 bytes) 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.255567133, 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:50:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:50:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:50:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.255101146000001, 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:50:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:50:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.254594597000001, 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:50:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:50:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:50:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.254298688999999, 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:50:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:50:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.254111676000001, 2024-04-23T23:50:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:50:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:50:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.253910006000002, 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:50:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:50:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.253769744, 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.253524008999999, 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:50:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:50:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.253246484000002, 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.253003912, 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:50:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:50:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:50:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.252825548000001, 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.252542956999999, 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.252350473, 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:50:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.251922531999998, 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:50:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:50:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.251770198999999, 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:50:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:50:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.251667116, 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:50:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:50:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.251350169999998, 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:50:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:50:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.251108743, 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.250884196000001, 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:50:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.250629299, 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:42.851 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:42.852 INFO:teuthology.orchestra.run.smithi092.stdout:4012 trunc from 0x19f9b76 to 0xb0861dd 2024-04-23T23:50:42.852 INFO:teuthology.orchestra.run.smithi092.stdout:4013 punch from 0x3c35b46 to 0x3c3f3b0, (0x986a bytes) 2024-04-23T23:50:42.852 INFO:teuthology.orchestra.run.smithi092.stdout:4014 write 0x6e2e080 thru 0x6e3cb91 (0xeb12 bytes) 2024-04-23T23:50:42.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:42.856 INFO:teuthology.orchestra.run.smithi092.stdout:4015 write 0x791eb5b thru 0x7923d93 (0x5239 bytes) 2024-04-23T23:50:42.856 INFO:teuthology.orchestra.run.smithi092.stdout:4016 write 0xaf8f605 thru 0xaf90e3f (0x183b bytes) 2024-04-23T23:50:42.856 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.856 INFO:teuthology.orchestra.run.smithi092.stdout:4018 read 0x72e8850 thru 0x72f4671 (0xbe22 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4019 writesame 0xe4626e7 thru 0xe468934 data_size 0x1(0x624e bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4020 punch from 0x7b23bb6 to 0x7b295ff, (0x5a49 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4021 writesame 0x640db6a thru 0x6413aec data_size 0x7(0x5f83 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4022 write 0x9a999d1 thru 0x9aa5791 (0xbdc1 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4024 write 0x9593106 thru 0x95a2fcb (0xfec6 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4025 trunc from 0xe468935 to 0xc4cbfef 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4026 read 0x33d5ef6 thru 0x33d89fd (0x2b08 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4027 trunc from 0xc4cbfef to 0xa825924 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4028 writesame 0x776119f thru 0x776619c data_size 0x3(0x4ffe bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4029 read 0x3282d5f thru 0x3286ce0 (0x3f82 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4030 read 0x2d435bf thru 0x2d4fc64 (0xc6a6 bytes) 2024-04-23T23:50:42.857 INFO:teuthology.orchestra.run.smithi092.stdout:4031 read 0x1dbc95b thru 0x1dbe951 (0x1ff7 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4035 trunc from 0xa825924 to 0x8e4c67c 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4036 punch from 0x109da9c to 0x10a0e55, (0x33b9 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4038 write 0xc8233b thru 0xc88fa2 (0x6c68 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4040 writesame 0x10d1c08 thru 0x10decbc data_size 0x17(0xd0b5 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4041 read 0x817d3cb thru 0x81829ff (0x5635 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4044 punch from 0x808c3b9 to 0x809342f, (0x7076 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4045 read 0x495e683 thru 0x4961afe (0x347c bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4046 writesame 0xf7e423 thru 0xf84b0b data_size 0xb(0x66e9 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4047 read 0x2cf5c1c thru 0x2cf6e50 (0x1235 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4050 read 0x57edb65 thru 0x57f9c4b (0xc0e7 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4051 writesame 0x288121 thru 0x293f60 data_size 0x20(0xbe40 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4052 write 0xbbaa11 thru 0xbc8a42 (0xe032 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4053 read 0x7ed98b7 thru 0x7ee7a9c (0xe1e6 bytes) 2024-04-23T23:50:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:4054 read 0x53fcf91 thru 0x53ff0ed (0x215d bytes) 2024-04-23T23:50:42.859 INFO:teuthology.orchestra.run.smithi092.stdout:4055 read 0x5dd0f74 thru 0x5ddc777 (0xb804 bytes) 2024-04-23T23:50:42.859 INFO:teuthology.orchestra.run.smithi092.stdout:4056 write 0xd55c844 thru 0xd564b19 (0x82d6 bytes) 2024-04-23T23:50:42.859 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:42.866+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:42.860 INFO:teuthology.orchestra.run.smithi092.stdout:4057 read 0x79ff2b1 thru 0x7a0707c (0x7dcc bytes) 2024-04-23T23:50:42.860 INFO:teuthology.orchestra.run.smithi092.stdout:4058 read 0x2fe2c9f thru 0x2fe3a9a (0xdfc bytes) 2024-04-23T23:50:42.860 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.860 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.860 INFO:teuthology.orchestra.run.smithi092.stdout:4061 write 0x2147941 thru 0x214f575 (0x7c35 bytes) 2024-04-23T23:50:42.862 INFO:teuthology.orchestra.run.smithi092.stdout:4062 write 0x33ab4b8 thru 0x33b66d0 (0xb219 bytes) 2024-04-23T23:50:42.865 INFO:teuthology.orchestra.run.smithi092.stdout:4063 punch from 0x5bb6654 to 0x5bb89a2, (0x234e bytes) 2024-04-23T23:50:42.865 INFO:teuthology.orchestra.run.smithi092.stdout:4064 write 0x2da0dc8 thru 0x2da5f35 (0x516e bytes) 2024-04-23T23:50:42.868 INFO:teuthology.orchestra.run.smithi092.stdout:4067 read 0x290f0bc thru 0x29154fe (0x6443 bytes) 2024-04-23T23:50:42.868 INFO:teuthology.orchestra.run.smithi092.stdout:4068 writesame 0x685bc5d thru 0x686ae2b data_size 0x1(0xf1cf bytes) 2024-04-23T23:50:42.908 INFO:teuthology.orchestra.run.smithi092.stdout:4070 writesame 0x952bab4 thru 0x952e089 data_size 0x2(0x25d6 bytes) 2024-04-23T23:50:42.914 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.914 INFO:teuthology.orchestra.run.smithi092.stdout:4073 punch from 0x62dfc4b to 0x62e5836, (0x5beb bytes) 2024-04-23T23:50:42.914 INFO:teuthology.orchestra.run.smithi092.stdout:4074 punch from 0x2a00c62 to 0x2a0f68d, (0xea2b bytes) 2024-04-23T23:50:42.921 INFO:teuthology.orchestra.run.smithi092.stdout:4076 write 0xe970ba5 thru 0xe97799c (0x6df8 bytes) 2024-04-23T23:50:42.924 INFO:teuthology.orchestra.run.smithi092.stdout:4077 writesame 0x7ddb9ae thru 0x7ddc4b5 data_size 0x4(0xb08 bytes) 2024-04-23T23:50:42.937 INFO:teuthology.orchestra.run.smithi092.stdout:4078 trunc from 0xe97799d to 0x58c6214 2024-04-23T23:50:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:4080 write 0x55cbfdf thru 0x55cea2c (0x2a4e bytes) 2024-04-23T23:50:42.953 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:42.967+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:42.956 INFO:teuthology.orchestra.run.smithi092.stdout:4081 trunc from 0x58c6214 to 0x7f3a8cf 2024-04-23T23:50:42.961 INFO:teuthology.orchestra.run.smithi092.stdout:4083 flatten 2024-04-23T23:50:42.966 INFO:teuthology.orchestra.run.smithi092.stdout:4084 read 0x4f9773f thru 0x4f99564 (0x1e26 bytes) 2024-04-23T23:50:42.967 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:42.967 INFO:teuthology.orchestra.run.smithi092.stdout:4086 read 0x3c46457 thru 0x3c4d08a (0x6c34 bytes) 2024-04-23T23:50:42.971 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:42.971 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:42.971 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:42.972 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:42.972 INFO:teuthology.orchestra.run.smithi092.stdout:4087 trunc from 0x7f3a8cf to 0x38e8d16 2024-04-23T23:50:42.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:42.982 INFO:teuthology.orchestra.run.smithi092.stdout:4088 read 0x2296989 thru 0x229d722 (0x6d9a bytes) 2024-04-23T23:50:42.982 INFO:teuthology.orchestra.run.smithi092.stdout:4089 read 0x1e7ce66 thru 0x1e881b8 (0xb353 bytes) 2024-04-23T23:50:42.982 INFO:teuthology.orchestra.run.smithi092.stdout:4090 read 0x15cae03 thru 0x15cf2b8 (0x44b6 bytes) 2024-04-23T23:50:42.982 INFO:teuthology.orchestra.run.smithi092.stdout:4091 writesame 0x139a3a2 thru 0x13a7672 data_size 0x1(0xd2d1 bytes) 2024-04-23T23:50:43.030 INFO:teuthology.orchestra.run.smithi092.stdout:4092 punch from 0x3702c37 to 0x3703c21, (0xfea bytes) 2024-04-23T23:50:43.031 INFO:teuthology.orchestra.run.smithi092.stdout:4093 read 0x28a6ce9 thru 0x28a7711 (0xa29 bytes) 2024-04-23T23:50:43.045 INFO:teuthology.orchestra.run.smithi092.stdout:4094 write 0xa4d22e5 thru 0xa4d3345 (0x1061 bytes) 2024-04-23T23:50:43.054 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:43.068+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:43.057 INFO:teuthology.orchestra.run.smithi092.stdout:4095 write 0xc548314 thru 0xc554cb4 (0xc9a1 bytes) 2024-04-23T23:50:43.061 INFO:teuthology.orchestra.run.smithi092.stdout:4096 read 0xc4c3a91 thru 0xc4c90c0 (0x5630 bytes) 2024-04-23T23:50:43.061 INFO:teuthology.orchestra.run.smithi092.stdout:4097 punch from 0x526d78b to 0x526dca3, (0x518 bytes) 2024-04-23T23:50:43.067 INFO:teuthology.orchestra.run.smithi092.stdout:4098 writesame 0xd748e3e thru 0xd750e27 data_size 0x7(0x7fea bytes) 2024-04-23T23:50:43.067 INFO:teuthology.orchestra.run.smithi092.stdout:4099 read 0x4a60e96 thru 0x4a6fdd1 (0xef3c bytes) 2024-04-23T23:50:43.081 INFO:teuthology.orchestra.run.smithi092.stdout:4100 trunc from 0xd750e28 to 0x2395974 2024-04-23T23:50:43.098 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:43.098 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:43.110 INFO:teuthology.orchestra.run.smithi092.stdout:4101 write 0xac529ff thru 0xac5b6b1 (0x8cb3 bytes) 2024-04-23T23:50:43.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:43.113 INFO:teuthology.orchestra.run.smithi092.stdout:4103 writesame 0x432e770 thru 0x433daff data_size 0x48(0xf390 bytes) 2024-04-23T23:50:43.116 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.116 INFO:teuthology.orchestra.run.smithi092.stdout:4106 punch from 0x690a911 to 0x6913363, (0x8a52 bytes) 2024-04-23T23:50:43.117 INFO:teuthology.orchestra.run.smithi092.stdout:4108 read 0x66431db thru 0x6652b8f (0xf9b5 bytes) 2024-04-23T23:50:43.117 INFO:teuthology.orchestra.run.smithi092.stdout:4110 punch from 0x906262b to 0x9065558, (0x2f2d bytes) 2024-04-23T23:50:43.117 INFO:teuthology.orchestra.run.smithi092.stdout:4111 writesame 0x3a68044 thru 0x3a69e3d data_size 0x3(0x1dfa bytes) 2024-04-23T23:50:43.120 INFO:teuthology.orchestra.run.smithi092.stdout:4112 read 0x5efcaac thru 0x5f04cd8 (0x822d bytes) 2024-04-23T23:50:43.121 INFO:teuthology.orchestra.run.smithi092.stdout:4113 write 0x487ff2 thru 0x493eeb (0xbefa bytes) 2024-04-23T23:50:43.125 INFO:teuthology.orchestra.run.smithi092.stdout:4115 trunc from 0xac5b6b2 to 0xd380cc5 2024-04-23T23:50:43.135 INFO:teuthology.orchestra.run.smithi092.stdout:4116 write 0xe1d72cb thru 0xe1da315 (0x304b bytes) 2024-04-23T23:50:43.139 INFO:teuthology.orchestra.run.smithi092.stdout:4117 write 0xd2678c8 thru 0xd271006 (0x973f bytes) 2024-04-23T23:50:43.143 INFO:teuthology.orchestra.run.smithi092.stdout:4118 write 0x356d287 thru 0x357cf72 (0xfcec bytes) 2024-04-23T23:50:43.146 INFO:teuthology.orchestra.run.smithi092.stdout:4119 write 0x3397f94 thru 0x339f599 (0x7606 bytes) 2024-04-23T23:50:43.149 INFO:teuthology.orchestra.run.smithi092.stdout:4121 read 0x2ea9cef thru 0x2eb2e79 (0x918b bytes) 2024-04-23T23:50:43.149 INFO:teuthology.orchestra.run.smithi092.stdout:4124 punch from 0x6b9faaf to 0x6ba9ca1, (0xa1f2 bytes) 2024-04-23T23:50:43.149 INFO:teuthology.orchestra.run.smithi092.stdout:4125 write 0x19278a0 thru 0x19281a7 (0x908 bytes) 2024-04-23T23:50:43.150 INFO:teuthology.orchestra.run.smithi092.stdout:4126 write 0x6d78316 thru 0x6d80dfd (0x8ae8 bytes) 2024-04-23T23:50:43.154 INFO:teuthology.orchestra.run.smithi092.stdout:4127 writesame 0x79e2a15 thru 0x79ef5ec data_size 0x2c(0xcbd8 bytes) 2024-04-23T23:50:43.154 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:43.168+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:43.154 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:50:43.155 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:43.155 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T23:50:43.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T23:50:43.158 INFO:teuthology.orchestra.run.smithi092.stdout:4128 read 0x9ad4bb5 thru 0x9ae4827 (0xfc73 bytes) 2024-04-23T23:50:43.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:50:43.168 INFO:teuthology.orchestra.run.smithi092.stdout:4129 trunc from 0xe1da316 to 0x61b7ab8 2024-04-23T23:50:43.195 INFO:teuthology.orchestra.run.smithi092.stdout:4130 write 0xe3022db thru 0xe303c99 (0x19bf bytes) 2024-04-23T23:50:43.198 INFO:teuthology.orchestra.run.smithi092.stdout:4132 write 0xbc319b6 thru 0xbc40fcb (0xf616 bytes) 2024-04-23T23:50:43.204 INFO:teuthology.orchestra.run.smithi092.stdout:4133 read 0x811733e thru 0x811c5f8 (0x52bb bytes) 2024-04-23T23:50:43.213 INFO:teuthology.orchestra.run.smithi092.stdout:4135 trunc from 0xe303c9a to 0x6c9e4a2 2024-04-23T23:50:43.225 INFO:teuthology.orchestra.run.smithi092.stdout:4137 read 0x64e8869 thru 0x64f52ad (0xca45 bytes) 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.788513612000003, 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:43.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:43.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.040854652, 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.198985346999997, 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:58072 3.9 e54/22 3:967d967a:::rbd_data.117966a158ce.0000000000000002:head v 54'280, mlcod=53'276)", 2024-04-23T23:50:43.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.054628485, 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003273686, 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193448+0000", 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193458+0000", 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0628000000000001e-05 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193469+0000", 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380000000000001e-05 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193479+0000", 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193529+0000", 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-05 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193551+0000", 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196689+0000", 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031378420000000001 2024-04-23T23:50:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196721+0000", 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2400000000000001e-05 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9790071170000001, 2024-04-23T23:50:43.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:43.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:50:43.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.259 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9292122299999992, 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:50:43.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:50:43.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9280814779999993, 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63551 3.9 e58/22 3:94220160:::rbd_data.11798ae5b3f1.000000000000004c:head v 58'303, mlcod=58'301)", 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8897380540000004, 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029639520000000002, 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:43.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6400000000000001e-07 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358354+0000", 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5396e-05 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358366+0000", 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2289e-05 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358373+0000", 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4150000000000004e-06 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358441+0000", 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8459000000000003e-05 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358462+0000", 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1078e-05 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.361257+0000", 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002794745 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.361302+0000", 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4805999999999998e-05 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8825871620000001, 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:50:43.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8798167879999994, 2024-04-23T23:50:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63748 3.4 e58/22 3:21ed4642:::rbd_data.11798ae5b3f1.0000000000000064:head v 58'132, mlcod=58'128)", 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8514257670000003, 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003193577, 2024-04-23T23:50:43.280 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396651+0000", 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396657+0000", 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8980000000000001e-06 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396667+0000", 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0288e-05 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396673+0000", 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396718+0000", 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396736+0000", 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8071e-05 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399806+0000", 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003069881 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399844+0000", 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8056999999999998e-05 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64075 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'141, mlcod=58'139)", 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7851148959999996, 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028754750000000002, 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:43.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462969+0000", 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9519999999999994e-06 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462982+0000", 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3095000000000001e-05 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462992+0000", 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8719999999999997e-06 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462999+0000", 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463049+0000", 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0837999999999997e-05 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463067+0000", 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7374e-05 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.465799+0000", 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027323080000000001 2024-04-23T23:50:43.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.465836+0000", 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7314999999999998e-05 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7844309589999998, 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:50:43.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7820128769999997, 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64292 3.9 e58/22 3:957b2643:::rbd_data.11798ae5b3f1.000000000000011d:head v 58'325, mlcod=58'323)", 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7185353590000005, 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003216596, 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529541+0000", 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529546+0000", 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6619999999999996e-06 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529557+0000", 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1035e-05 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529564+0000", 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0219999999999999e-06 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529615+0000", 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0899000000000002e-05 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529636+0000", 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1739999999999999e-05 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532716+0000", 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003079755 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532757+0000", 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64347 3.4 e58/22 3:23ee6249:::rbd_data.11798ae5b3f1.000000000000013a:head v 58'151, mlcod=58'149)", 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6997227660000007, 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036492059999999999, 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548354+0000", 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6899999999999996e-07 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548359+0000", 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548372+0000", 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3050999999999999e-05 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548379+0000", 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548448+0000", 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9326999999999995e-05 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548466+0000", 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659000000000002e-05 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551946+0000", 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034799850000000001 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552002+0000", 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6508999999999997e-05 2024-04-23T23:50:43.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64341 3.0 e58/22 3:0635a7b0:::rbd_data.11798ae5b3f1.0000000000000138:head v 58'169, mlcod=58'166)", 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6993148629999997, 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032676770000000001, 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548766+0000", 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548777+0000", 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1453e-05 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548784+0000", 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6460000000000004e-06 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548833+0000", 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8498000000000002e-05 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548852+0000", 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9142000000000001e-05 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551992+0000", 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031404649999999998 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552029+0000", 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6381999999999998e-05 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 58/22 rep_tid 964 entries 58'170 (0'0) error 3:0535656d:::rbd_data.11798ae5b3f1.000000000000013d:head by client.4473.0:64343 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 58'166)", 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.698993776, 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029602270000000002, 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8800000000000003e-07 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549090+0000", 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3470000000000001e-06 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549106+0000", 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6222000000000002e-05 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549120+0000", 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4260000000000001e-05 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549190+0000", 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0184000000000004e-05 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552042+0000", 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028517260000000002 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6917204950000002, 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:43.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:67439 3.0 e58/22 3:0718c4b2:::rbd_data.117911508d31.0000000000000006:head v 58'177, mlcod=58'175)", 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3389519320000005, 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036877949999999998, 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:43.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909125+0000", 2024-04-23T23:50:43.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909535+0000", 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040998899999999998 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909550+0000", 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5092000000000001e-05 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909558+0000", 2024-04-23T23:50:43.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6790000000000005e-06 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909610+0000", 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1996000000000002e-05 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909638+0000", 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8231000000000001e-05 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912774+0000", 2024-04-23T23:50:43.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031359740000000001 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912812+0000", 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7710999999999999e-05 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:43.322 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:43.323 INFO:teuthology.orchestra.run.smithi092.stdout:4138 trunc from 0x6c9e4a2 to 0xeb82ba6 2024-04-23T23:50:43.323 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.323 INFO:teuthology.orchestra.run.smithi092.stdout:4142 punch from 0x495912e to 0x495cb02, (0x39d4 bytes) 2024-04-23T23:50:43.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:43.326 INFO:teuthology.orchestra.run.smithi092.stdout:4143 trunc from 0xeb82ba6 to 0x3726846 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4144 read 0x26357e3 thru 0x263b8ba (0x60d8 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4147 write 0x9a28834 thru 0x9a2c221 (0x39ee bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4148 write 0x89a92a9 thru 0x89b16b8 (0x8410 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4151 punch from 0x51698d5 to 0x517548d, (0xbbb8 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4153 read 0x667219 thru 0x676020 (0xee08 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4154 punch from 0x85051ee to 0x851060e, (0xb420 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4155 punch from 0x1860256 to 0x186b749, (0xb4f3 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4156 writesame 0x5e75e50 thru 0x5e7b7c5 data_size 0x21(0x5976 bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4158 read 0x245f60 thru 0x24d4fc (0x759d bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4160 read 0x7fac42b thru 0x7fb9336 (0xcf0c bytes) 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:4164 read 0x7966346 thru 0x7968cf1 (0x29ac bytes) 2024-04-23T23:50:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:4167 punch from 0x82db159 to 0x82de3f1, (0x3298 bytes) 2024-04-23T23:50:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:4168 read 0x926d072 thru 0x9279b0b (0xca9a bytes) 2024-04-23T23:50:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:4169 write 0xcb44380 thru 0xcb49b9c (0x581d bytes) 2024-04-23T23:50:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:4170 trunc from 0xcb49b9d to 0x8ea524b 2024-04-23T23:50:43.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:43.330 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:43.330 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:43.330 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:43.330 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:43.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T23:50:43.331 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:43.340+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:43.338 INFO:teuthology.orchestra.run.smithi092.stdout:4171 writesame 0xdd03b89 thru 0xdd07b30 data_size 0x54(0x3fa8 bytes) 2024-04-23T23:50:43.338 INFO:teuthology.orchestra.run.smithi092.stdout:4172 write 0x6e80454 thru 0x6e8b545 (0xb0f2 bytes) 2024-04-23T23:50:43.342 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.348 INFO:teuthology.orchestra.run.smithi092.stdout:4174 writesame 0xe5968d8 thru 0xe5a4953 data_size 0x6(0xe07c bytes) 2024-04-23T23:50:43.348 INFO:teuthology.orchestra.run.smithi092.stdout:4175 read 0xb4c50e1 thru 0xb4c6452 (0x1372 bytes) 2024-04-23T23:50:43.353 INFO:teuthology.orchestra.run.smithi092.stdout:4176 trunc from 0xe5a4954 to 0xa3b1191 2024-04-23T23:50:43.359 INFO:teuthology.orchestra.run.smithi092.stdout:4177 read 0x44c1d05 thru 0x44c6e98 (0x5194 bytes) 2024-04-23T23:50:43.370 INFO:teuthology.orchestra.run.smithi092.stdout:4178 writesame 0xe1cf7b9 thru 0xe1de990 data_size 0x8e(0xf1d8 bytes) 2024-04-23T23:50:43.370 INFO:teuthology.orchestra.run.smithi092.stdout:4180 write 0x4b4503a thru 0x4b48de0 (0x3da7 bytes) 2024-04-23T23:50:43.373 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.373 INFO:teuthology.orchestra.run.smithi092.stdout:4182 read 0xae95ffe thru 0xae99a55 (0x3a58 bytes) 2024-04-23T23:50:43.373 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.373 INFO:teuthology.orchestra.run.smithi092.stdout:4185 write 0xc6d35e6 thru 0xc6d4ba3 (0x15be bytes) 2024-04-23T23:50:43.376 INFO:teuthology.orchestra.run.smithi092.stdout:4186 writesame 0x1dd1d4c thru 0x1dd93fe data_size 0x7(0x76b3 bytes) 2024-04-23T23:50:43.380 INFO:teuthology.orchestra.run.smithi092.stdout:4189 write 0x2ccea0e thru 0x2cd26a2 (0x3c95 bytes) 2024-04-23T23:50:43.385 INFO:teuthology.orchestra.run.smithi092.stdout:4190 read 0xb88bdf4 thru 0xb89b750 (0xf95d bytes) 2024-04-23T23:50:43.385 INFO:teuthology.orchestra.run.smithi092.stdout:4191 writesame 0x54b2840 thru 0x54ba72e data_size 0x5(0x7eef bytes) 2024-04-23T23:50:43.393 INFO:teuthology.orchestra.run.smithi092.stdout:4192 trunc from 0xe1de991 to 0xc22a9df 2024-04-23T23:50:43.401 INFO:teuthology.orchestra.run.smithi092.stdout:4193 read 0x392ad85 thru 0x392e6d9 (0x3955 bytes) 2024-04-23T23:50:43.402 INFO:teuthology.orchestra.run.smithi092.stdout:4195 punch from 0x35dbf37 to 0x35e6c33, (0xacfc bytes) 2024-04-23T23:50:43.409 INFO:teuthology.orchestra.run.smithi092.stdout:4196 trunc from 0xc22a9df to 0x5f534c7 2024-04-23T23:50:43.421 INFO:teuthology.orchestra.run.smithi092.stdout:4197 writesame 0xe6b886 thru 0xe6bd32 data_size 0x13(0x4ad bytes) 2024-04-23T23:50:43.422 INFO:teuthology.orchestra.run.smithi092.stdout:4198 writesame 0x4bebc5 thru 0x4c3120 data_size 0x2e(0x455c bytes) 2024-04-23T23:50:43.425 INFO:teuthology.orchestra.run.smithi092.stdout:4199 read 0x217fc21 thru 0x21859e5 (0x5dc5 bytes) 2024-04-23T23:50:43.425 INFO:teuthology.orchestra.run.smithi092.stdout:4201 read 0x3aad053 thru 0x3aad873 (0x821 bytes) 2024-04-23T23:50:43.425 INFO:teuthology.orchestra.run.smithi092.stdout:4202 write 0x37891ab thru 0x3798ad2 (0xf928 bytes) 2024-04-23T23:50:43.427 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:43.441+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:43.433 INFO:teuthology.orchestra.run.smithi092.stdout:4203 trunc from 0x5f534c7 to 0x14084a3 2024-04-23T23:50:43.434 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:43.434 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:43.434 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:43.434 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:43.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:43.448 INFO:teuthology.orchestra.run.smithi092.stdout:4205 read 0x11ddb54 thru 0x11e9ac0 (0xbf6d bytes) 2024-04-23T23:50:43.467 INFO:teuthology.orchestra.run.smithi092.stdout:4206 writesame 0xc181ee0 thru 0xc191da1 data_size 0x1(0xfec2 bytes) 2024-04-23T23:50:43.468 INFO:teuthology.orchestra.run.smithi092.stdout:4207 writesame 0x3602d5e thru 0x360b2e2 data_size 0x13(0x8585 bytes) 2024-04-23T23:50:43.471 INFO:teuthology.orchestra.run.smithi092.stdout:4208 write 0x30ef238 thru 0x30f9e69 (0xac32 bytes) 2024-04-23T23:50:43.484 INFO:teuthology.orchestra.run.smithi092.stdout:4209 trunc from 0xc191da2 to 0x3d44370 2024-04-23T23:50:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:4210 write 0x6c3272e thru 0x6c3a13f (0x7a12 bytes) 2024-04-23T23:50:43.501 INFO:teuthology.orchestra.run.smithi092.stdout:4213 punch from 0x1137949 to 0x1147543, (0xfbfa bytes) 2024-04-23T23:50:43.501 INFO:teuthology.orchestra.run.smithi092.stdout:4214 read 0x345d7e0 thru 0x3460ddc (0x35fd bytes) 2024-04-23T23:50:43.501 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.515 INFO:teuthology.orchestra.run.smithi092.stdout:4216 write 0xd7a35f7 thru 0xd7b28c1 (0xf2cb bytes) 2024-04-23T23:50:43.518 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.518 INFO:teuthology.orchestra.run.smithi092.stdout:4219 read 0xcd1e49d thru 0xcd2e307 (0xfe6b bytes) 2024-04-23T23:50:43.519 INFO:teuthology.orchestra.run.smithi092.stdout:4220 write 0x3fee1f4 thru 0x3ff37cf (0x55dc bytes) 2024-04-23T23:50:43.522 INFO:teuthology.orchestra.run.smithi092.stdout:4221 write 0xf89e73 thru 0xf8ea18 (0x4ba6 bytes) 2024-04-23T23:50:43.523 INFO:teuthology.orchestra.run.smithi092.stdout:4222 writesame 0x98040ec thru 0x980b9ab data_size 0xa1(0x78c0 bytes) 2024-04-23T23:50:43.526 INFO:teuthology.orchestra.run.smithi092.stdout:4224 writesame 0x2b8415d thru 0x2b89a5e data_size 0x1(0x5902 bytes) 2024-04-23T23:50:43.527 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:43.528 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:43.528 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:43.528 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:43.528 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:43.528 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:43.528 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:43.528 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:43.541+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:43.530 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:43.531 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:43.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T23:50:43.544 INFO:teuthology.orchestra.run.smithi092.stdout:4225 trunc from 0xd7b28c2 to 0x991caf2 2024-04-23T23:50:43.567 INFO:teuthology.orchestra.run.smithi092.stdout:4226 trunc from 0x991caf2 to 0xf22118 2024-04-23T23:50:43.581 INFO:teuthology.orchestra.run.smithi092.stdout:4228 read 0x61200e thru 0x62111b (0xf10e bytes) 2024-04-23T23:50:43.581 INFO:teuthology.orchestra.run.smithi092.stdout:4230 read 0x6bcd0c thru 0x6c7163 (0xa458 bytes) 2024-04-23T23:50:43.581 INFO:teuthology.orchestra.run.smithi092.stdout:4231 write 0xa82f92 thru 0xa88644 (0x56b3 bytes) 2024-04-23T23:50:43.583 INFO:teuthology.orchestra.run.smithi092.stdout:4233 read 0x394589 thru 0x3a0734 (0xc1ac bytes) 2024-04-23T23:50:43.583 INFO:teuthology.orchestra.run.smithi092.stdout:4234 read 0xae3ee9 thru 0xaebf84 (0x809c bytes) 2024-04-23T23:50:43.584 INFO:teuthology.orchestra.run.smithi092.stdout:4235 read 0x37af51 thru 0x3861f6 (0xb2a6 bytes) 2024-04-23T23:50:43.584 INFO:teuthology.orchestra.run.smithi092.stdout:4238 write 0x9b9c89 thru 0x9bc329 (0x26a1 bytes) 2024-04-23T23:50:43.591 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:43.591 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:43.591 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:43.591 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:43.591 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:43.599 INFO:teuthology.orchestra.run.smithi092.stdout:4239 trunc from 0xf22118 to 0xbc52309 2024-04-23T23:50:43.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:43.608 INFO:teuthology.orchestra.run.smithi092.stdout:4240 trunc from 0xbc52309 to 0xc79167a 2024-04-23T23:50:43.613 INFO:teuthology.orchestra.run.smithi092.stdout:4241 write 0x15e499f thru 0x15f3258 (0xe8ba bytes) 2024-04-23T23:50:43.617 INFO:teuthology.orchestra.run.smithi092.stdout:4243 read 0x17538c5 thru 0x1758a09 (0x5145 bytes) 2024-04-23T23:50:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:4244 trunc from 0xc79167a to 0xbd09ea4 2024-04-23T23:50:43.638 INFO:teuthology.orchestra.run.smithi092.stdout:4245 trunc from 0xbd09ea4 to 0x46d30e 2024-04-23T23:50:43.644 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:43.658+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:43.652 INFO:teuthology.orchestra.run.smithi092.stdout:4248 read 0x60153 thru 0x7001b (0xfec9 bytes) 2024-04-23T23:50:43.665 INFO:teuthology.orchestra.run.smithi092.stdout:4249 writesame 0x4fa19b6 thru 0x4faa705 data_size 0x1c(0x8d50 bytes) 2024-04-23T23:50:43.665 INFO:teuthology.orchestra.run.smithi092.stdout:4250 read 0x218f94e thru 0x2197cad (0x8360 bytes) 2024-04-23T23:50:43.665 INFO:teuthology.orchestra.run.smithi092.stdout:4251 read 0x11de9d8 thru 0x11eb82b (0xce54 bytes) 2024-04-23T23:50:43.678 INFO:teuthology.orchestra.run.smithi092.stdout:4252 write 0xab26e9d thru 0xab3505f (0xe1c3 bytes) 2024-04-23T23:50:43.682 INFO:teuthology.orchestra.run.smithi092.stdout:4253 read 0x7368ab9 thru 0x7376db4 (0xe2fc bytes) 2024-04-23T23:50:43.682 INFO:teuthology.orchestra.run.smithi092.stdout:4254 read 0xaa45ef3 thru 0xaa4c469 (0x6577 bytes) 2024-04-23T23:50:43.687 INFO:teuthology.orchestra.run.smithi092.stdout:4255 write 0xaeaafbf thru 0xaeaee04 (0x3e46 bytes) 2024-04-23T23:50:43.691 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:43.691 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:43.691 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:43.691 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:43.695 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:43.695 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:43.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T23:50:43.697 INFO:teuthology.orchestra.run.smithi092.stdout:4256 trunc from 0xaeaee05 to 0x39445e8 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.293029853, 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:50:43.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:50:43.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.293019973, 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:50:43.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.292809626, 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:50:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.724 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.290978287000002, 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:50:43.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:50:43.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.288908656, 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776818420000001, 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:50:43.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776792764, 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:50:43.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:50:43.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776725274, 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:50:43.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:50:43.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776680212999999, 2024-04-23T23:50:43.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776643712999999, 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:50:43.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:50:43.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776626753999999, 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.758 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.77654807, 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:50:43.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:50:43.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:50:43.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776481037, 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:50:43.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.767 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.776347436000002, 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:50:43.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.775928755999999, 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:50:43.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.775778114000001, 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:50:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.775652732000001, 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:50:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:43.784 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.775437215, 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:50:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:50:43.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.775199880999999, 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:50:43.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:50:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.774959985999999, 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:43.793 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:43.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:50:43.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:50:43.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:43.797 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:43.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:4257 trunc from 0x39445e8 to 0xc48bd92 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:4258 trunc from 0xc48bd92 to 0x31e6519 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:4263 trunc from 0x31e6519 to 0x563e891 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:4264 punch from 0x1f1aa15 to 0x1f249c4, (0x9faf bytes) 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:4266 writesame 0xb81e0b5 thru 0xb81f063 data_size 0xb(0xfaf bytes) 2024-04-23T23:50:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:4268 write 0x854ac8a thru 0x8551961 (0x6cd8 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4269 writesame 0x8a37e52 thru 0x8a434ef data_size 0x55(0xb69e bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4270 read 0xa9ed48e thru 0xa9f1111 (0x3c84 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4271 write 0x8f539be thru 0x8f5e223 (0xa866 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4272 read 0x209f332 thru 0x20aa16d (0xae3c bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4274 writesame 0x456b86d thru 0x4574238 data_size 0x2(0x89cc bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4275 write 0x82b10e1 thru 0x82be738 (0xd658 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4277 read 0x58dd251 thru 0x58e46e7 (0x7497 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4278 write 0x4b62c05 thru 0x4b6d2e1 (0xa6dd bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4280 writesame 0xe277da7 thru 0xe28779a data_size 0x22(0xf9f4 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4281 read 0x3ebd1ff thru 0x3ec507d (0x7e7f bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4282 read 0x10f9912 thru 0x1106ecd (0xd5bc bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4283 read 0xaf105f0 thru 0xaf166cf (0x60e0 bytes) 2024-04-23T23:50:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:4284 trunc from 0xe28779b to 0x88643f8 2024-04-23T23:50:43.803 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:43.812+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:43.820 INFO:teuthology.orchestra.run.smithi092.stdout:4285 write 0xd3a796d thru 0xd3aa832 (0x2ec6 bytes) 2024-04-23T23:50:43.823 INFO:teuthology.orchestra.run.smithi092.stdout:4286 punch from 0xcae9561 to 0xcaefa56, (0x64f5 bytes) 2024-04-23T23:50:43.823 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.823 INFO:teuthology.orchestra.run.smithi092.stdout:4289 read 0x53d8427 thru 0x53e7b93 (0xf76d bytes) 2024-04-23T23:50:43.823 INFO:teuthology.orchestra.run.smithi092.stdout:4290 read 0x8546ecb thru 0x8552b5d (0xbc93 bytes) 2024-04-23T23:50:43.823 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.823 INFO:teuthology.orchestra.run.smithi092.stdout:4297 write 0x66214dd thru 0x6630b08 (0xf62c bytes) 2024-04-23T23:50:43.827 INFO:teuthology.orchestra.run.smithi092.stdout:4298 writesame 0x150d17e thru 0x151abea data_size 0x39(0xda6d bytes) 2024-04-23T23:50:43.831 INFO:teuthology.orchestra.run.smithi092.stdout:4299 punch from 0x445ee71 to 0x446466b, (0x57fa bytes) 2024-04-23T23:50:43.831 INFO:teuthology.orchestra.run.smithi092.stdout:4300 write 0x8e97a63 thru 0x8e98549 (0xae7 bytes) 2024-04-23T23:50:43.834 INFO:teuthology.orchestra.run.smithi092.stdout:4301 write 0xa981569 thru 0xa98a105 (0x8b9d bytes) 2024-04-23T23:50:43.837 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.837 INFO:teuthology.orchestra.run.smithi092.stdout:4304 write 0xadb1a95 thru 0xadb9917 (0x7e83 bytes) 2024-04-23T23:50:43.840 INFO:teuthology.orchestra.run.smithi092.stdout:4305 read 0xc287ab thru 0xc2eb4b (0x63a1 bytes) 2024-04-23T23:50:43.842 INFO:teuthology.orchestra.run.smithi092.stdout:4307 punch from 0x3f88288 to 0x3f8f5a9, (0x7321 bytes) 2024-04-23T23:50:43.842 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.842 INFO:teuthology.orchestra.run.smithi092.stdout:4309 read 0x254c6e9 thru 0x255a4d3 (0xddeb bytes) 2024-04-23T23:50:43.842 INFO:teuthology.orchestra.run.smithi092.stdout:4311 write 0x9467eb2 thru 0x94717ea (0x9939 bytes) 2024-04-23T23:50:43.844 INFO:teuthology.orchestra.run.smithi092.stdout:4312 write 0xc3174ad thru 0xc3252a9 (0xddfd bytes) 2024-04-23T23:50:43.845 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:43.845 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:43.845 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:43.845 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:43.850 INFO:teuthology.orchestra.run.smithi092.stdout:4313 write 0x9d9faa2 thru 0x9da050c (0xa6b bytes) 2024-04-23T23:50:43.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:43.851 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:43.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-04-23T23:50:43.851 INFO:teuthology.orchestra.run.smithi092.stdout:4314 read 0x6a35036 thru 0x6a3ddf3 (0x8dbe bytes) 2024-04-23T23:50:43.851 INFO:teuthology.orchestra.run.smithi092.stdout:4316 punch from 0x72d1a62 to 0x72d920d, (0x77ab bytes) 2024-04-23T23:50:43.852 INFO:teuthology.orchestra.run.smithi092.stdout:4318 trunc from 0xd3aa833 to 0xe8e5378 2024-04-23T23:50:43.857 INFO:teuthology.orchestra.run.smithi092.stdout:4319 write 0x48e0ceb thru 0x48efc0f (0xef25 bytes) 2024-04-23T23:50:43.859 INFO:teuthology.orchestra.run.smithi092.stdout:4320 punch from 0x9abcbff to 0x9ac9155, (0xc556 bytes) 2024-04-23T23:50:43.859 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.859 INFO:teuthology.orchestra.run.smithi092.stdout:4322 read 0x7af1a2d thru 0x7afec5e (0xd232 bytes) 2024-04-23T23:50:43.859 INFO:teuthology.orchestra.run.smithi092.stdout:4324 punch from 0xa933af6 to 0xa9394ea, (0x59f4 bytes) 2024-04-23T23:50:43.859 INFO:teuthology.orchestra.run.smithi092.stdout:4325 writesame 0x36973f2 thru 0x369d70a data_size 0x1(0x6319 bytes) 2024-04-23T23:50:43.881 INFO:teuthology.orchestra.run.smithi092.stdout:4326 write 0xe968af1 thru 0xe96c196 (0x36a6 bytes) 2024-04-23T23:50:43.884 INFO:teuthology.orchestra.run.smithi092.stdout:4327 write 0x8eb0ab thru 0x8f8655 (0xd5ab bytes) 2024-04-23T23:50:43.886 INFO:teuthology.orchestra.run.smithi092.stdout:4329 writesame 0x9d21262 thru 0x9d2d98a data_size 0x67(0xc729 bytes) 2024-04-23T23:50:43.887 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:43.889 INFO:teuthology.orchestra.run.smithi092.stdout:4334 trunc from 0xe96c197 to 0xd168110 2024-04-23T23:50:43.899 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:43.913+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:43.900 INFO:teuthology.orchestra.run.smithi092.stdout:4335 write 0xa1de760 thru 0xa1e94dc (0xad7d bytes) 2024-04-23T23:50:43.903 INFO:teuthology.orchestra.run.smithi092.stdout:4336 read 0x7ad2443 thru 0x7ad2f1b (0xad9 bytes) 2024-04-23T23:50:43.904 INFO:teuthology.orchestra.run.smithi092.stdout:4337 punch from 0x4e34eb3 to 0x4e37eb2, (0x2fff bytes) 2024-04-23T23:50:43.904 INFO:teuthology.orchestra.run.smithi092.stdout:4338 read 0x332d1d3 thru 0x332ea91 (0x18bf bytes) 2024-04-23T23:50:43.914 INFO:teuthology.orchestra.run.smithi092.stdout:4339 trunc from 0xd168110 to 0x45ef836 2024-04-23T23:50:43.929 INFO:teuthology.orchestra.run.smithi092.stdout:4340 read 0x28cc1aa thru 0x28d6ab9 (0xa910 bytes) 2024-04-23T23:50:43.929 INFO:teuthology.orchestra.run.smithi092.stdout:4341 writesame 0x2988ccb thru 0x2995ced data_size 0x1(0xd023 bytes) 2024-04-23T23:50:43.949 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:43.950 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:43.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:43.970 INFO:teuthology.orchestra.run.smithi092.stdout:4342 trunc from 0x45ef836 to 0xae039af 2024-04-23T23:50:43.977 INFO:teuthology.orchestra.run.smithi092.stdout:4343 writesame 0x9600ae9 thru 0x9609269 data_size 0x3(0x8781 bytes) 2024-04-23T23:50:43.986 INFO:teuthology.orchestra.run.smithi092.stdout:4344 read 0x3d574aa thru 0x3d607a3 (0x92fa bytes) 2024-04-23T23:50:43.987 INFO:teuthology.orchestra.run.smithi092.stdout:4345 writesame 0xdd735b thru 0xde4b18 data_size 0x69(0xd7be bytes) 2024-04-23T23:50:43.989 INFO:teuthology.orchestra.run.smithi092.stdout:4346 read 0x901d921 thru 0x9028d13 (0xb3f3 bytes) 2024-04-23T23:50:43.994 INFO:teuthology.orchestra.run.smithi092.stdout:4348 trunc from 0xae039af to 0x69f9e35 2024-04-23T23:50:44.000 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:44.014+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:44.004 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:44.004 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:44.004 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:44.004 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:44.004 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:44.004 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:44.005 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:44.005 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:44.005 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:44.005 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:44.005 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:44.005 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:44.006 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.006 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.006 INFO:teuthology.orchestra.run.smithi092.stdout:4351 punch from 0x5d3ac62 to 0x5d47012, (0xc3b0 bytes) 2024-04-23T23:50:44.007 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:44.007 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:44.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-04-23T23:50:44.012 INFO:teuthology.orchestra.run.smithi092.stdout:4352 trunc from 0x69f9e35 to 0xb945cfd 2024-04-23T23:50:44.029 INFO:teuthology.orchestra.run.smithi092.stdout:4353 write 0xe74f6f2 thru 0xe754a56 (0x5365 bytes) 2024-04-23T23:50:44.033 INFO:teuthology.orchestra.run.smithi092.stdout:4355 writesame 0xde0998e thru 0xde0d0df data_size 0x49(0x3752 bytes) 2024-04-23T23:50:44.053 INFO:teuthology.orchestra.run.smithi092.stdout:4356 trunc from 0xe754a57 to 0x4304ab 2024-04-23T23:50:44.066 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.066 INFO:teuthology.orchestra.run.smithi092.stdout:4359 read 0x50512 thru 0x58392 (0x7e81 bytes) 2024-04-23T23:50:44.066 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:44.066 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:44.066 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:44.067 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:44.067 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:44.071 INFO:teuthology.orchestra.run.smithi092.stdout:4360 write 0x915369 thru 0x921090 (0xbd28 bytes) 2024-04-23T23:50:44.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:44.084 INFO:teuthology.orchestra.run.smithi092.stdout:4365 write 0x4a26316 thru 0x4a2a968 (0x4653 bytes) 2024-04-23T23:50:44.087 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.087 INFO:teuthology.orchestra.run.smithi092.stdout:4369 write 0x3916574 thru 0x3918420 (0x1ead bytes) 2024-04-23T23:50:44.097 INFO:teuthology.orchestra.run.smithi092.stdout:4371 trunc from 0x4a2a969 to 0xb29c1b8 2024-04-23T23:50:44.101 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:44.115+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:44.103 INFO:teuthology.orchestra.run.smithi092.stdout:4374 write 0x335cc6a thru 0x3361ba3 (0x4f3a bytes) 2024-04-23T23:50:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:4375 read 0x77bf25f thru 0x77c54f7 (0x6299 bytes) 2024-04-23T23:50:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:4376 punch from 0xb1dc80a to 0xb1dd05a, (0x850 bytes) 2024-04-23T23:50:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:4378 read 0xa20678d thru 0xa208493 (0x1d07 bytes) 2024-04-23T23:50:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:4380 read 0xa4f322a thru 0xa4f9fa6 (0x6d7d bytes) 2024-04-23T23:50:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:4381 punch from 0x3eb73f2 to 0x3eb7a6f, (0x67d bytes) 2024-04-23T23:50:44.115 INFO:teuthology.orchestra.run.smithi092.stdout:4382 trunc from 0xb29c1b8 to 0x40d71c0 2024-04-23T23:50:44.122 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.122 INFO:teuthology.orchestra.run.smithi092.stdout:4384 read 0x4059f07 thru 0x40668ad (0xc9a7 bytes) 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:4385 writesame 0x9ae69d6 thru 0x9aee1fd data_size 0x14(0x7828 bytes) 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:4386 read 0x824da97 thru 0x8257a2b (0x9f95 bytes) 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:4388 read 0x7f9a6eb thru 0x7f9a703 (0x19 bytes) 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:4390 read 0x26495f8 thru 0x264f425 (0x5e2e bytes) 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:4391 read 0x1a15226 thru 0x1a1a9ef (0x57ca bytes) 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.135 INFO:teuthology.orchestra.run.smithi092.stdout:4393 write 0x444abb8 thru 0x4456470 (0xb8b9 bytes) 2024-04-23T23:50:44.138 INFO:teuthology.orchestra.run.smithi092.stdout:4394 writesame 0x9987402 thru 0x999153c data_size 0x19(0xa13b bytes) 2024-04-23T23:50:44.142 INFO:teuthology.orchestra.run.smithi092.stdout:4395 read 0x44872a1 thru 0x449488e (0xd5ee bytes) 2024-04-23T23:50:44.142 INFO:teuthology.orchestra.run.smithi092.stdout:4397 read 0xe3ed5f thru 0xe49766 (0xaa08 bytes) 2024-04-23T23:50:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:4398 trunc from 0x9aee1fe to 0xbbd4cdf 2024-04-23T23:50:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:4399 read 0x589734f thru 0x589f80a (0x84bc bytes) 2024-04-23T23:50:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:4401 read 0x5a2d8e4 thru 0x5a31f57 (0x4674 bytes) 2024-04-23T23:50:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:4402 punch from 0xa502b5a to 0xa51106f, (0xe515 bytes) 2024-04-23T23:50:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:4404 write 0x4b39397 thru 0x4b488bb (0xf525 bytes) 2024-04-23T23:50:44.154 INFO:teuthology.orchestra.run.smithi092.stdout:4406 write 0x114865f thru 0x114b36d (0x2d0f bytes) 2024-04-23T23:50:44.156 INFO:teuthology.orchestra.run.smithi092.stdout:4407 writesame 0x8823982 thru 0x882e7d6 data_size 0x1(0xae55 bytes) 2024-04-23T23:50:44.162 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:44.166 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:44.167 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:44.167 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:44.167 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:44.167 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:44.167 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:44.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:44.170 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:44.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-04-23T23:50:44.183 INFO:teuthology.orchestra.run.smithi092.stdout:4408 read 0x1b4c3ed thru 0x1b5af06 (0xeb1a bytes) 2024-04-23T23:50:44.184 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.184 INFO:teuthology.orchestra.run.smithi092.stdout:4410 writesame 0x32be57e thru 0x32c3f9e data_size 0x1(0x5a21 bytes) 2024-04-23T23:50:44.186 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:44.186 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.930039803, 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:50:44.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:50:44.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:44.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.930018923, 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:50:44.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.924161137, 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:50:44.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.924146889999999, 2024-04-23T23:50:44.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:50:44.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.922112957, 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:50:44.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:50:44.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.922101014999999, 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.920123881, 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:50:44.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.920095797, 2024-04-23T23:50:44.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:50:44.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.920083262, 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:50:44.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:50:44.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.918142174, 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:50:44.228 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.918130828000001, 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.233 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:44.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:50:44.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:50:44.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.918119974, 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:44.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:50:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:50:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.918108683, 2024-04-23T23:50:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:50:44.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.916069910000001, 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:50:44.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.916053927, 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.916033307999999, 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:50:44.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:50:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.91601331, 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:50:44.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:50:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:50:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.915995029999999, 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:50:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:50:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.914992831999999, 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.268 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:44.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.914981333, 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:50:44.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:50:44.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.276 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:44.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:44.279 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:44.291+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4412 punch from 0x576da96 to 0x577606c, (0x85d6 bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4413 writesame 0x14b6135 thru 0x14c0e72 data_size 0x19(0xad3e bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4414 trunc from 0xbbd4cdf to 0x5ae6a8f 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4415 read 0x409431b thru 0x409cb57 (0x883d bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4417 punch from 0x24e1a8d to 0x24f04f5, (0xea68 bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4418 punch from 0x24a99a7 to 0x24aea45, (0x509e bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4419 read 0x30ade41 thru 0x30b394e (0x5b0e bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4420 writesame 0x7d2321c thru 0x7d27599 data_size 0x35(0x437e bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4423 read 0x3826a35 thru 0x3832e82 (0xc44e bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4424 write 0xe5c8476 thru 0xe5d15a7 (0x9132 bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4426 write 0xed4c4fc thru 0xed52d80 (0x6885 bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4427 writesame 0x310db22 thru 0x310fc4f data_size 0x1f(0x212e bytes) 2024-04-23T23:50:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:4428 read 0x9a59f0f thru 0x9a66bf5 (0xcce7 bytes) 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:4430 write 0x40be38d thru 0x40c2948 (0x45bc bytes) 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:4431 punch from 0xa62f694 to 0xa638592, (0x8efe bytes) 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:4432 write 0x65427 thru 0x69a30 (0x460a bytes) 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:4435 writesame 0x1df9dc thru 0x1ea2f1 data_size 0x17(0xa916 bytes) 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:4437 punch from 0x6f79bf3 to 0x6f81c38, (0x8045 bytes) 2024-04-23T23:50:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:4438 trunc from 0xed52d81 to 0xd395298 2024-04-23T23:50:44.288 INFO:teuthology.orchestra.run.smithi092.stdout:4439 trunc from 0xd395298 to 0x4c983b1 2024-04-23T23:50:44.296 INFO:teuthology.orchestra.run.smithi092.stdout:4440 read 0x1d4c405 thru 0x1d572d9 (0xaed5 bytes) 2024-04-23T23:50:44.296 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.296 INFO:teuthology.orchestra.run.smithi092.stdout:4442 writesame 0x243d846 thru 0x244a6d4 data_size 0x1(0xce8f bytes) 2024-04-23T23:50:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:44.334 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:44.335 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:44.335 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:44.335 INFO:teuthology.orchestra.run.smithi092.stdout:4450 trunc from 0x4c983b1 to 0xa5e379f 2024-04-23T23:50:44.337 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:44.337 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:44.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-04-23T23:50:44.341 INFO:teuthology.orchestra.run.smithi092.stdout:4451 trunc from 0xa5e379f to 0x9a99573 2024-04-23T23:50:44.353 INFO:teuthology.orchestra.run.smithi092.stdout:4452 trunc from 0x9a99573 to 0x557660b 2024-04-23T23:50:44.357 INFO:teuthology.orchestra.run.smithi092.stdout:4453 clone 29 order 18 su 4096 sc 2 2024-04-23T23:50:44.379 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:44.393+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:44.423 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:44.424 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:44.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:44.472 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:44.473 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:44.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:44.475 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:44.517 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:44.531+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:44.574 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:44.574 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:44.574 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:44.574 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:44.574 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:44.580 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:44.595+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:44.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:44.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.684 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.178396665000001, 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:44.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:50:44.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.689 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.178363616999999, 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.694 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.178341117999999, 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.695 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:50:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.176471200999998, 2024-04-23T23:50:44.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:50:44.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.176442725000001, 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:50:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:50:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.176420280999999, 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:50:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:50:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:50:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.174464553, 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.174445425999998, 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:50:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:50:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.174419277999998, 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.174398585999999, 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.174376859999999, 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:50:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.172479970000001, 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.172455523, 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.172438429, 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.170461048, 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:50:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.756 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.170442170000001, 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:50:44.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.168461130000001, 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:44.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:50:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.168421328000001, 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:50:44.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.769 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.168399304000001, 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.773 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.168377762999999, 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:50:44.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:44.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:44.778 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:44.778 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:44.716+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:44.779 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:44.779 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:44.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:44.782 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:44.796+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:44.795 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:44.812 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:44.828 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:44.828 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:44.845 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:44.846 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:44.862 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:44.862 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:44.880 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:44.881 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:44.883 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:44.897+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:44.884 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:44.884 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:44.884 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:44.885 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:44.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:44.898 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:44.984 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:44.998+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:45.034 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.034 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:45.086 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:45.100+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:45.144 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.144 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:45.197 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:45.211+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:45.262 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.262 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:45.285 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-04-23T23:50:45.286 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:45.300+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:45.387 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:45.401+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:45.401 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.401 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:45.488 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:45.502+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:45.562 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.562 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:45.609 INFO:teuthology.orchestra.run.smithi092.stdout:4454 write 0xd1457a6 thru 0xd14d6e0 (0x7f3b bytes) 2024-04-23T23:50:45.609 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:45.623+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:45.617 INFO:teuthology.orchestra.run.smithi092.stdout:4455 punch from 0x1268fd7 to 0x126f10d, (0x6136 bytes) 2024-04-23T23:50:45.617 INFO:teuthology.orchestra.run.smithi092.stdout:4456 write 0xaa383a1 thru 0xaa389ea (0x64a bytes) 2024-04-23T23:50:45.619 INFO:teuthology.orchestra.run.smithi092.stdout:4457 write 0xa21c309 thru 0xa22813b (0xbe33 bytes) 2024-04-23T23:50:45.624 INFO:teuthology.orchestra.run.smithi092.stdout:4459 writesame 0xbb2738d thru 0xbb2dc0b data_size 0x25(0x687f bytes) 2024-04-23T23:50:45.637 INFO:teuthology.orchestra.run.smithi092.stdout:4460 trunc from 0xd14d6e1 to 0x539f9f3 2024-04-23T23:50:45.660 INFO:teuthology.orchestra.run.smithi092.stdout:4461 trunc from 0x539f9f3 to 0xdaf9bae 2024-04-23T23:50:45.666 INFO:teuthology.orchestra.run.smithi092.stdout:4462 write 0x77f5d46 thru 0x77f8f98 (0x3253 bytes) 2024-04-23T23:50:45.669 INFO:teuthology.orchestra.run.smithi092.stdout:4463 write 0xa407224 thru 0xa410179 (0x8f56 bytes) 2024-04-23T23:50:45.673 INFO:teuthology.orchestra.run.smithi092.stdout:4465 writesame 0x413a8ff thru 0x41405d0 data_size 0x2(0x5cd2 bytes) 2024-04-23T23:50:45.688 INFO:teuthology.orchestra.run.smithi092.stdout:4466 read 0x3b6225d thru 0x3b64f6a (0x2d0e bytes) 2024-04-23T23:50:45.689 INFO:teuthology.orchestra.run.smithi092.stdout:4467 read 0x62eab0f thru 0x62fa55d (0xfa4f bytes) 2024-04-23T23:50:45.689 INFO:teuthology.orchestra.run.smithi092.stdout:4468 write 0x3e9e5c1 thru 0x3ea6ddd (0x881d bytes) 2024-04-23T23:50:45.690 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:45.704+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:45.695 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.696 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.698 INFO:teuthology.orchestra.run.smithi092.stdout:4469 flatten 2024-04-23T23:50:45.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:45.794 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:45.807+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:45.892 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:45.906+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:45.918 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:45.918 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:45.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:45.993 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:46.007+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:46.094 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:46.107+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:46.120 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:46.120 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:46.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:46.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:50:46.194 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:46.208+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:46.295 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:46.309+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:46.345 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:46.345 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:46.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:46.400 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:46.414+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:4470 write 0xde16578 thru 0xde260f6 (0xfb7f bytes) 2024-04-23T23:50:46.487 INFO:teuthology.orchestra.run.smithi092.stdout:4471 trunc from 0xde260f7 to 0xa0256b1 2024-04-23T23:50:46.498 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:46.498 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:46.512+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:46.501 INFO:teuthology.orchestra.run.smithi092.stdout:4474 trunc from 0xa0256b1 to 0xc543157 2024-04-23T23:50:46.506 INFO:teuthology.orchestra.run.smithi092.stdout:4476 write 0xb2ec727 thru 0xb2f2bfa (0x64d4 bytes) 2024-04-23T23:50:46.511 INFO:teuthology.orchestra.run.smithi092.stdout:4477 read 0x50f6a68 thru 0x510444d (0xd9e6 bytes) 2024-04-23T23:50:46.511 INFO:teuthology.orchestra.run.smithi092.stdout:4478 writesame 0x1954542 thru 0x1957b64 data_size 0x1(0x3623 bytes) 2024-04-23T23:50:46.522 INFO:teuthology.orchestra.run.smithi092.stdout:4479 read 0x48974e9 thru 0x489b97c (0x4494 bytes) 2024-04-23T23:50:46.522 INFO:teuthology.orchestra.run.smithi092.stdout:4480 writesame 0x3ecb8b thru 0x3f9cd9 data_size 0x35(0xd14f bytes) 2024-04-23T23:50:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:4481 writesame 0xca50e98 thru 0xca51825 data_size 0x1(0x98e bytes) 2024-04-23T23:50:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:4485 read 0x5761bfe thru 0x5768893 (0x6c96 bytes) 2024-04-23T23:50:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:4486 write 0xb8089af thru 0xb8169dc (0xe02e bytes) 2024-04-23T23:50:46.541 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:46.541 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:46.542 INFO:teuthology.orchestra.run.smithi092.stdout:4487 trunc from 0xca51826 to 0x7fe188f 2024-04-23T23:50:46.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:46.552 INFO:teuthology.orchestra.run.smithi092.stdout:4488 read 0x5b01d1d thru 0x5b0db6c (0xbe50 bytes) 2024-04-23T23:50:46.553 INFO:teuthology.orchestra.run.smithi092.stdout:4490 punch from 0x5772fd8 to 0x577ecbb, (0xbce3 bytes) 2024-04-23T23:50:46.561 INFO:teuthology.orchestra.run.smithi092.stdout:4491 write 0xbb85422 thru 0xbb943f4 (0xefd3 bytes) 2024-04-23T23:50:46.567 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:46.567 INFO:teuthology.orchestra.run.smithi092.stdout:4494 writesame 0x5aaf065 thru 0x5ab53af data_size 0x25(0x634b bytes) 2024-04-23T23:50:46.574 INFO:teuthology.orchestra.run.smithi092.stdout:4495 write 0x3b0f3bf thru 0x3b1ca62 (0xd6a4 bytes) 2024-04-23T23:50:46.580 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:46.580 INFO:teuthology.orchestra.run.smithi092.stdout:4497 punch from 0x95a0316 to 0x95adc94, (0xd97e bytes) 2024-04-23T23:50:46.580 INFO:teuthology.orchestra.run.smithi092.stdout:4498 write 0x78f1992 thru 0x78f1fb9 (0x628 bytes) 2024-04-23T23:50:46.582 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:46.582 INFO:teuthology.orchestra.run.smithi092.stdout:4501 clone 30 order 25 su 524288 sc 8 2024-04-23T23:50:46.599 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:46.612+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:46.665 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:46.665 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:46.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:46.702 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:46.716+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:46.804 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:46.817+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:46.816 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:46.817 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:46.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:46.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:46.910 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:46.923+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:46.970 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:46.970 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:46.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:47.007 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:47.020+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:47.030 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:47.030 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:47.048 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:47.048 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:47.065 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:47.065 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:47.082 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:50:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.639874081000002, 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.892215121, 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:50:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.050345815999997, 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:50:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:50:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:50:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:58072 3.9 e54/22 3:967d967a:::rbd_data.117966a158ce.0000000000000002:head v 54'280, mlcod=53'276)", 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.905988954000001, 2024-04-23T23:50:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003273686, 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193448+0000", 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193458+0000", 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0628000000000001e-05 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193469+0000", 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380000000000001e-05 2024-04-23T23:50:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193479+0000", 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193529+0000", 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-05 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193551+0000", 2024-04-23T23:50:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196689+0000", 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031378420000000001 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196721+0000", 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2400000000000001e-05 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.830367585999999, 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:50:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:50:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.780572699, 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.779441947, 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:50:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:50:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63551 3.9 e58/22 3:94220160:::rbd_data.11798ae5b3f1.000000000000004c:head v 58'303, mlcod=58'301)", 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.741098523, 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029639520000000002, 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358338+0000", 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6400000000000001e-07 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358354+0000", 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5396e-05 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358366+0000", 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2289e-05 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358373+0000", 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4150000000000004e-06 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358441+0000", 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8459000000000003e-05 2024-04-23T23:50:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.358462+0000", 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1078e-05 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.361257+0000", 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002794745 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.361302+0000", 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4805999999999998e-05 2024-04-23T23:50:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.733947630999999, 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.731177257000001, 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63748 3.4 e58/22 3:21ed4642:::rbd_data.11798ae5b3f1.0000000000000064:head v 58'132, mlcod=58'128)", 2024-04-23T23:50:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.702786236, 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003193577, 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396651+0000", 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396657+0000", 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8980000000000001e-06 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396667+0000", 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0288e-05 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396673+0000", 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396718+0000", 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396736+0000", 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8071e-05 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399806+0000", 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003069881 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399844+0000", 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8056999999999998e-05 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64075 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'141, mlcod=58'139)", 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.636475365000001, 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028754750000000002, 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462961+0000", 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462969+0000", 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9519999999999994e-06 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462982+0000", 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3095000000000001e-05 2024-04-23T23:50:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462992+0000", 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8719999999999997e-06 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.462999+0000", 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463049+0000", 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0837999999999997e-05 2024-04-23T23:50:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463067+0000", 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7374e-05 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.465799+0000", 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027323080000000001 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.465836+0000", 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7314999999999998e-05 2024-04-23T23:50:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.635791427999999, 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:50:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.633373345999999, 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64292 3.9 e58/22 3:957b2643:::rbd_data.11798ae5b3f1.000000000000011d:head v 58'325, mlcod=58'323)", 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.569895828, 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003216596, 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529541+0000", 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529546+0000", 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6619999999999996e-06 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529557+0000", 2024-04-23T23:50:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1035e-05 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529564+0000", 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0219999999999999e-06 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529615+0000", 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0899000000000002e-05 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529636+0000", 2024-04-23T23:50:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1739999999999999e-05 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532716+0000", 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003079755 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532757+0000", 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64347 3.4 e58/22 3:23ee6249:::rbd_data.11798ae5b3f1.000000000000013a:head v 58'151, mlcod=58'149)", 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.551083235, 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036492059999999999, 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548354+0000", 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6899999999999996e-07 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548359+0000", 2024-04-23T23:50:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548372+0000", 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3050999999999999e-05 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548379+0000", 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548448+0000", 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9326999999999995e-05 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548466+0000", 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659000000000002e-05 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551946+0000", 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034799850000000001 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552002+0000", 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6508999999999997e-05 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64341 3.0 e58/22 3:0635a7b0:::rbd_data.11798ae5b3f1.0000000000000138:head v 58'169, mlcod=58'166)", 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.550675332000001, 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032676770000000001, 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548766+0000", 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548777+0000", 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1453e-05 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548784+0000", 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6460000000000004e-06 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548833+0000", 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8498000000000002e-05 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548852+0000", 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9142000000000001e-05 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551992+0000", 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031404649999999998 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552029+0000", 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6381999999999998e-05 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 58/22 rep_tid 964 entries 58'170 (0'0) error 3:0535656d:::rbd_data.11798ae5b3f1.000000000000013d:head by client.4473.0:64343 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 58'166)", 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.550354244999999, 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029602270000000002, 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549082+0000", 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8800000000000003e-07 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549090+0000", 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3470000000000001e-06 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549106+0000", 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6222000000000002e-05 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549120+0000", 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4260000000000001e-05 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.549190+0000", 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0184000000000004e-05 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552042+0000", 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028517260000000002 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.163 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.543080964, 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:50:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:50:47.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:67439 3.0 e58/22 3:0718c4b2:::rbd_data.117911508d31.0000000000000006:head v 58'177, mlcod=58'175)", 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.190312401, 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036877949999999998, 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909125+0000", 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909535+0000", 2024-04-23T23:50:47.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040998899999999998 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909550+0000", 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5092000000000001e-05 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909558+0000", 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6790000000000005e-06 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909610+0000", 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1996000000000002e-05 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909638+0000", 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8231000000000001e-05 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912774+0000", 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031359740000000001 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:47.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912812+0000", 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7710999999999999e-05 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:47.173 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:47.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:47.175 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:47.176 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:47.176 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:47.176 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:47.176 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:47.177 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:47.177 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:47.178 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:47.178 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:50:47.179 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:47.188+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:47.275 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:47.289+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:47.285 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:47.285 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:47.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:47.353 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-04-23T23:50:47.386 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:47.400+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:47.439 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:47.439 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:47.439 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:47.439 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:47.440 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:47.448 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:47.466 INFO:teuthology.orchestra.run.smithi092.stdout:4503 trunc from 0xbb943f5 to 0x723a881 2024-04-23T23:50:47.487 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:47.501+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:47.498 INFO:teuthology.orchestra.run.smithi092.stdout:4504 read 0x95404c thru 0x95ac5e (0x6c13 bytes) 2024-04-23T23:50:47.506 INFO:teuthology.orchestra.run.smithi092.stdout:4505 write 0x954eeea thru 0x9556138 (0x724f bytes) 2024-04-23T23:50:47.508 INFO:teuthology.orchestra.run.smithi092.stdout:4506 read 0x856f146 thru 0x857b8be (0xc779 bytes) 2024-04-23T23:50:47.513 INFO:teuthology.orchestra.run.smithi092.stdout:4507 trunc from 0x9556139 to 0x5b44e9e 2024-04-23T23:50:47.519 INFO:teuthology.orchestra.run.smithi092.stdout:4508 read 0x4a0ced3 thru 0x4a1cc0f (0xfd3d bytes) 2024-04-23T23:50:47.535 INFO:teuthology.orchestra.run.smithi092.stdout:4509 writesame 0xe34f701 thru 0xe35f3c0 data_size 0x60(0xfcc0 bytes) 2024-04-23T23:50:47.536 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.536 INFO:teuthology.orchestra.run.smithi092.stdout:4512 punch from 0x4cd115a to 0x4cd8606, (0x74ac bytes) 2024-04-23T23:50:47.537 INFO:teuthology.orchestra.run.smithi092.stdout:4514 write 0x95cce1c thru 0x95d6ee5 (0xa0ca bytes) 2024-04-23T23:50:47.538 INFO:teuthology.orchestra.run.smithi092.stdout:4515 trunc from 0xe35f3c1 to 0xe613283 2024-04-23T23:50:47.543 INFO:teuthology.orchestra.run.smithi092.stdout:4516 read 0xc820a81 thru 0xc82eabf (0xe03f bytes) 2024-04-23T23:50:47.543 INFO:teuthology.orchestra.run.smithi092.stdout:4517 write 0x9f5ea7d thru 0x9f64d59 (0x62dd bytes) 2024-04-23T23:50:47.545 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.545 INFO:teuthology.orchestra.run.smithi092.stdout:4519 punch from 0xe4abb6e to 0xe4bb054, (0xf4e6 bytes) 2024-04-23T23:50:47.545 INFO:teuthology.orchestra.run.smithi092.stdout:4520 writesame 0x51f0ec8 thru 0x51f662d data_size 0x21(0x5766 bytes) 2024-04-23T23:50:47.547 INFO:teuthology.orchestra.run.smithi092.stdout:4521 punch from 0x5a0fbe8 to 0x5a19c74, (0xa08c bytes) 2024-04-23T23:50:47.547 INFO:teuthology.orchestra.run.smithi092.stdout:4522 write 0x7aa06c thru 0x7b941e (0xf3b3 bytes) 2024-04-23T23:50:47.548 INFO:teuthology.orchestra.run.smithi092.stdout:4524 punch from 0x3b951e0 to 0x3b9ed35, (0x9b55 bytes) 2024-04-23T23:50:47.548 INFO:teuthology.orchestra.run.smithi092.stdout:4526 punch from 0xe0f4864 to 0xe0f597f, (0x111b bytes) 2024-04-23T23:50:47.549 INFO:teuthology.orchestra.run.smithi092.stdout:4528 write 0xe0f8ff2 thru 0xe104bd0 (0xbbdf bytes) 2024-04-23T23:50:47.550 INFO:teuthology.orchestra.run.smithi092.stdout:4530 punch from 0x73d5898 to 0x73dd4e5, (0x7c4d bytes) 2024-04-23T23:50:47.550 INFO:teuthology.orchestra.run.smithi092.stdout:4531 read 0x8b68147 thru 0x8b763b6 (0xe270 bytes) 2024-04-23T23:50:47.551 INFO:teuthology.orchestra.run.smithi092.stdout:4532 read 0x9899181 thru 0x98a0217 (0x7097 bytes) 2024-04-23T23:50:47.551 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.551 INFO:teuthology.orchestra.run.smithi092.stdout:4535 write 0x3678f97 thru 0x36856da (0xc744 bytes) 2024-04-23T23:50:47.552 INFO:teuthology.orchestra.run.smithi092.stdout:4536 read 0x138909c thru 0x138a456 (0x13bb bytes) 2024-04-23T23:50:47.552 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.552 INFO:teuthology.orchestra.run.smithi092.stdout:4539 punch from 0x8ef8163 to 0x8ef9cb9, (0x1b56 bytes) 2024-04-23T23:50:47.552 INFO:teuthology.orchestra.run.smithi092.stdout:4540 read 0x396dda8 thru 0x396f012 (0x126b bytes) 2024-04-23T23:50:47.553 INFO:teuthology.orchestra.run.smithi092.stdout:4541 read 0xce29b10 thru 0xce38f20 (0xf411 bytes) 2024-04-23T23:50:47.553 INFO:teuthology.orchestra.run.smithi092.stdout:4542 write 0x8001595 thru 0x800f0ab (0xdb17 bytes) 2024-04-23T23:50:47.555 INFO:teuthology.orchestra.run.smithi092.stdout:4543 punch from 0x6565c65 to 0x656b552, (0x58ed bytes) 2024-04-23T23:50:47.555 INFO:teuthology.orchestra.run.smithi092.stdout:4544 write 0xd787280 thru 0xd7885e3 (0x1364 bytes) 2024-04-23T23:50:47.556 INFO:teuthology.orchestra.run.smithi092.stdout:4545 writesame 0x3c16571 thru 0x3c20805 data_size 0x1(0xa295 bytes) 2024-04-23T23:50:47.562 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.144096537999999, 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.144086657999999, 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:50:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:50:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:50:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.143876311, 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:50:47.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.142044972000001, 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:50:47.577 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:50:47.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:50:47.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.139975341, 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:50:47.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627885105000001, 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.586 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:50:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:50:47.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:50:47.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:50:47.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:50:47.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:50:47.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:50:47.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:50:47.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:50:47.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.593 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.594 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.594 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:47.594 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627859448999999, 2024-04-23T23:50:47.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:50:47.594 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.594 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.595 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:50:47.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:47.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:47.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:50:47.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627791959, 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:47.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:50:47.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627746898000002, 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.612 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627710398000001, 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:50:47.617 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627693439000002, 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:50:47.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.623 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627614755, 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:50:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.628 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627547721999999, 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:50:47.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:50:47.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.627414121000001, 2024-04-23T23:50:47.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:50:47.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.626995440999998, 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:50:47.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.626844799000001, 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:50:47.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:50:47.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.626719417, 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.650 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.655 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.626503899999999, 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:50:47.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:50:47.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.626266565999998, 2024-04-23T23:50:47.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.626026670999998, 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:50:47.665 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:47.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:50:47.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:50:47.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:47.671 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:47.671 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:47.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:47.675 INFO:teuthology.orchestra.run.smithi092.stdout:4546 write 0x63e1e54 thru 0x63e51f2 (0x339f bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4548 read 0xe113bac thru 0xe1220c2 (0xe517 bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4549 trunc from 0xe613283 to 0x3d392d4 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4550 read 0x1f4d804 thru 0x1f58858 (0xb055 bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4552 punch from 0x1f62d3f to 0x1f6b16f, (0x8430 bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4555 trunc from 0x3d392d4 to 0xc418317 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4556 write 0xaab1a83 thru 0xaabfbdc (0xe15a bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4557 read 0xbb4aad8 thru 0xbb54896 (0x9dbf bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4558 trunc from 0xc418317 to 0x47b6221 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4561 write 0x20062ed thru 0x200ee60 (0x8b74 bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4563 punch from 0xd642b4 to 0xd707fc, (0xc548 bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4564 punch from 0x3d76996 to 0x3d85ec3, (0xf52d bytes) 2024-04-23T23:50:47.676 INFO:teuthology.orchestra.run.smithi092.stdout:4565 trunc from 0x47b6221 to 0x5e27cc3 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4566 punch from 0x3b88379 to 0x3b97622, (0xf2a9 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4567 read 0xf9f76d thru 0xfa473f (0x4fd3 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4568 write 0x6f5fee0 thru 0x6f61190 (0x12b1 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4569 write 0x9ab5e9c thru 0x9abf567 (0x96cc bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4571 punch from 0x887d415 to 0x888b697, (0xe282 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4572 read 0x1a3900 thru 0x1b187d (0xdf7e bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4573 writesame 0x58d170e thru 0x58d435f data_size 0x1f(0x2c52 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4579 read 0x7885109 thru 0x788d25b (0x8153 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4581 punch from 0x7736faa to 0x773a44d, (0x34a3 bytes) 2024-04-23T23:50:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:4582 trunc from 0x9abf568 to 0xb32a95f 2024-04-23T23:50:47.678 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:47.686+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:47.678 INFO:teuthology.orchestra.run.smithi092.stdout:4583 read 0x8fadd94 thru 0x8fbda04 (0xfc71 bytes) 2024-04-23T23:50:47.678 INFO:teuthology.orchestra.run.smithi092.stdout:4584 punch from 0x15ff5c0 to 0x160e029, (0xea69 bytes) 2024-04-23T23:50:47.678 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.679 INFO:teuthology.orchestra.run.smithi092.stdout:4587 trunc from 0xb32a95f to 0xdc91794 2024-04-23T23:50:47.684 INFO:teuthology.orchestra.run.smithi092.stdout:4588 punch from 0x1946e44 to 0x1952694, (0xb850 bytes) 2024-04-23T23:50:47.684 INFO:teuthology.orchestra.run.smithi092.stdout:4589 write 0x5f8dbba thru 0x5f8e99c (0xde3 bytes) 2024-04-23T23:50:47.686 INFO:teuthology.orchestra.run.smithi092.stdout:4590 writesame 0x9905e1 thru 0x99d9d9 data_size 0x1(0xd3f9 bytes) 2024-04-23T23:50:47.717 INFO:teuthology.orchestra.run.smithi092.stdout:4592 trunc from 0xdc91794 to 0xa2e0c2a 2024-04-23T23:50:47.726 INFO:teuthology.orchestra.run.smithi092.stdout:4593 read 0x5b49fdd thru 0x5b54aa8 (0xaacc bytes) 2024-04-23T23:50:47.727 INFO:teuthology.orchestra.run.smithi092.stdout:4595 punch from 0x4b07ed0 to 0x4b13036, (0xb166 bytes) 2024-04-23T23:50:47.727 INFO:teuthology.orchestra.run.smithi092.stdout:4596 writesame 0x92ffc02 thru 0x93043a2 data_size 0x1(0x47a1 bytes) 2024-04-23T23:50:47.739 INFO:teuthology.orchestra.run.smithi092.stdout:4599 writesame 0x1ef4f51 thru 0x1f015cc data_size 0x2(0xc67c bytes) 2024-04-23T23:50:47.756 INFO:teuthology.orchestra.run.smithi092.stdout:4600 read 0x7324cdb thru 0x7325c53 (0xf79 bytes) 2024-04-23T23:50:47.764 INFO:teuthology.orchestra.run.smithi092.stdout:4601 writesame 0xbecc55a thru 0xbed9fc0 data_size 0x1(0xda67 bytes) 2024-04-23T23:50:47.765 INFO:teuthology.orchestra.run.smithi092.stdout:4602 write 0x822d333 thru 0x822fdda (0x2aa8 bytes) 2024-04-23T23:50:47.772 INFO:teuthology.orchestra.run.smithi092.stdout:4603 trunc from 0xbed9fc1 to 0x695000d 2024-04-23T23:50:47.773 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:47.787+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:47.783 INFO:teuthology.orchestra.run.smithi092.stdout:4605 trunc from 0x695000d to 0x6284362 2024-04-23T23:50:47.786 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:47.786 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:47.789 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:47.798 INFO:teuthology.orchestra.run.smithi092.stdout:4608 write 0x8703c8e thru 0x8712e16 (0xf189 bytes) 2024-04-23T23:50:47.806 INFO:teuthology.orchestra.run.smithi092.stdout:4610 trunc from 0x8712e17 to 0x24b4fd3 2024-04-23T23:50:47.813 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.828 INFO:teuthology.orchestra.run.smithi092.stdout:4612 write 0xa86b632 thru 0xa877ea7 (0xc876 bytes) 2024-04-23T23:50:47.830 INFO:teuthology.orchestra.run.smithi092.stdout:4613 writesame 0xa64054b thru 0xa64c770 data_size 0x1(0xc226 bytes) 2024-04-23T23:50:47.854 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.855 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.855 INFO:teuthology.orchestra.run.smithi092.stdout:4617 read 0x4da2ef1 thru 0x4da6679 (0x3789 bytes) 2024-04-23T23:50:47.855 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.856 INFO:teuthology.orchestra.run.smithi092.stdout:4619 read 0xa45d660 thru 0xa4693e3 (0xbd84 bytes) 2024-04-23T23:50:47.863 INFO:teuthology.orchestra.run.smithi092.stdout:4620 write 0xb652774 thru 0xb65f4df (0xcd6c bytes) 2024-04-23T23:50:47.864 INFO:teuthology.orchestra.run.smithi092.stdout:4623 writesame 0x82a7c24 thru 0x82b4b4a data_size 0x3(0xcf27 bytes) 2024-04-23T23:50:47.875 INFO:teuthology.orchestra.run.smithi092.stdout:4625 read 0x9e999ac thru 0x9ea3301 (0x9956 bytes) 2024-04-23T23:50:47.879 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:47.893+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:4626 write 0xe1a3ba1 thru 0xe1a9788 (0x5be8 bytes) 2024-04-23T23:50:47.886 INFO:teuthology.orchestra.run.smithi092.stdout:4627 writesame 0x4cec5f8 thru 0x4cf4891 data_size 0x49(0x829a bytes) 2024-04-23T23:50:47.888 INFO:teuthology.orchestra.run.smithi092.stdout:4630 read 0x135ad75 thru 0x135ba2b (0xcb7 bytes) 2024-04-23T23:50:47.888 INFO:teuthology.orchestra.run.smithi092.stdout:4632 write 0x903821d thru 0x9040b96 (0x897a bytes) 2024-04-23T23:50:47.890 INFO:teuthology.orchestra.run.smithi092.stdout:4634 write 0x736c302 thru 0x7377198 (0xae97 bytes) 2024-04-23T23:50:47.892 INFO:teuthology.orchestra.run.smithi092.stdout:4635 write 0x1089bac thru 0x109718e (0xd5e3 bytes) 2024-04-23T23:50:47.895 INFO:teuthology.orchestra.run.smithi092.stdout:4637 trunc from 0xe1a9789 to 0xc86cb00 2024-04-23T23:50:47.904 INFO:teuthology.orchestra.run.smithi092.stdout:4638 write 0xb47ff35 thru 0xb4888d8 (0x89a4 bytes) 2024-04-23T23:50:47.906 INFO:teuthology.orchestra.run.smithi092.stdout:4639 punch from 0xb693e4e to 0xb6a1211, (0xd3c3 bytes) 2024-04-23T23:50:47.906 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.906 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.906 INFO:teuthology.orchestra.run.smithi092.stdout:4643 write 0x72d4c74 thru 0x72d6dfe (0x218b bytes) 2024-04-23T23:50:47.907 INFO:teuthology.orchestra.run.smithi092.stdout:4644 write 0x696d7e3 thru 0x6976002 (0x8820 bytes) 2024-04-23T23:50:47.920 INFO:teuthology.orchestra.run.smithi092.stdout:4645 trunc from 0xc86cb00 to 0x35541e3 2024-04-23T23:50:47.942 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:47.942 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:47.942 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:47.943 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:47.943 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:4646 write 0xebac3d9 thru 0xebb2f24 (0x6b4c bytes) 2024-04-23T23:50:47.948 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.948 INFO:teuthology.orchestra.run.smithi092.stdout:4650 read 0xb1fdf0a thru 0xb1ff0b8 (0x11af bytes) 2024-04-23T23:50:47.948 INFO:teuthology.orchestra.run.smithi092.stdout:4651 punch from 0x1f72685 to 0x1f81b14, (0xf48f bytes) 2024-04-23T23:50:47.948 INFO:teuthology.orchestra.run.smithi092.stdout:4652 write 0xb465029 thru 0xb46cd84 (0x7d5c bytes) 2024-04-23T23:50:47.950 INFO:teuthology.orchestra.run.smithi092.stdout:4653 punch from 0x70950c8 to 0x7097d89, (0x2cc1 bytes) 2024-04-23T23:50:47.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:47.959 INFO:teuthology.orchestra.run.smithi092.stdout:4654 trunc from 0xebb2f25 to 0x6d8705b 2024-04-23T23:50:47.967 INFO:teuthology.orchestra.run.smithi092.stdout:4655 read 0x5f7ef1 thru 0x5fde47 (0x5f57 bytes) 2024-04-23T23:50:47.971 INFO:teuthology.orchestra.run.smithi092.stdout:4656 trunc from 0x6d8705b to 0x3ca9728 2024-04-23T23:50:47.974 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:47.988+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:47.988 INFO:teuthology.orchestra.run.smithi092.stdout:4657 trunc from 0x3ca9728 to 0xcb7e709 2024-04-23T23:50:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:4658 punch from 0x2e349dc to 0x2e375ba, (0x2bde bytes) 2024-04-23T23:50:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:4659 punch from 0x38cbd57 to 0x38cc9e0, (0xc89 bytes) 2024-04-23T23:50:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:4662 read 0x6f74fac thru 0x6f7c3e1 (0x7436 bytes) 2024-04-23T23:50:47.995 INFO:teuthology.orchestra.run.smithi092.stdout:4663 read 0x5fbedc6 thru 0x5fc013c (0x1377 bytes) 2024-04-23T23:50:48.008 INFO:teuthology.orchestra.run.smithi092.stdout:4664 trunc from 0xcb7e709 to 0x11ff9a9 2024-04-23T23:50:48.027 INFO:teuthology.orchestra.run.smithi092.stdout:4665 write 0x5d5dbf2 thru 0x5d69fa6 (0xc3b5 bytes) 2024-04-23T23:50:48.029 INFO:teuthology.orchestra.run.smithi092.stdout:4666 punch from 0x508878a to 0x5090efc, (0x8772 bytes) 2024-04-23T23:50:48.029 INFO:teuthology.orchestra.run.smithi092.stdout:4671 read 0x35aa0e9 thru 0x35ad271 (0x3189 bytes) 2024-04-23T23:50:48.029 INFO:teuthology.orchestra.run.smithi092.stdout:4672 read 0x45d5ddd thru 0x45db5bd (0x57e1 bytes) 2024-04-23T23:50:48.029 INFO:teuthology.orchestra.run.smithi092.stdout:4673 write 0x4475159 thru 0x4478383 (0x322b bytes) 2024-04-23T23:50:48.031 INFO:teuthology.orchestra.run.smithi092.stdout:4674 writesame 0x55bc1d8 thru 0x55bc44c data_size 0x1(0x275 bytes) 2024-04-23T23:50:48.033 INFO:teuthology.orchestra.run.smithi092.stdout:4675 read 0x4d20f7e thru 0x4d2a3fd (0x9480 bytes) 2024-04-23T23:50:48.040 INFO:teuthology.orchestra.run.smithi092.stdout:4677 write 0x6e6dc36 thru 0x6e761ed (0x85b8 bytes) 2024-04-23T23:50:48.042 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.042 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.042 INFO:teuthology.orchestra.run.smithi092.stdout:4683 read 0x56ec613 thru 0x56f4ac2 (0x84b0 bytes) 2024-04-23T23:50:48.042 INFO:teuthology.orchestra.run.smithi092.stdout:4684 punch from 0x6e66a5f to 0x6e72890, (0xbe31 bytes) 2024-04-23T23:50:48.055 INFO:teuthology.orchestra.run.smithi092.stdout:4685 writesame 0xc6f9eb0 thru 0xc7084fa data_size 0xd(0xe64b bytes) 2024-04-23T23:50:48.056 INFO:teuthology.orchestra.run.smithi092.stdout:4686 punch from 0x83202e6 to 0x832ec1f, (0xe939 bytes) 2024-04-23T23:50:48.056 INFO:teuthology.orchestra.run.smithi092.stdout:4687 punch from 0x3e9747a to 0x3e989bc, (0x1542 bytes) 2024-04-23T23:50:48.063 INFO:teuthology.orchestra.run.smithi092.stdout:4688 write 0xd9c76f0 thru 0xd9d6eee (0xf7ff bytes) 2024-04-23T23:50:48.072 INFO:teuthology.orchestra.run.smithi092.stdout:4689 trunc from 0xd9d6eef to 0x7941ad6 2024-04-23T23:50:48.074 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:48.088+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:4691 trunc from 0x7941ad6 to 0x13377f1 2024-04-23T23:50:48.094 INFO:teuthology.orchestra.run.smithi092.stdout:4692 read 0x1cb5c7 thru 0x1d7bf9 (0xc633 bytes) 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.846116868999999, 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:50:48.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.846095988999998, 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:50:48.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:48.114 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.840238202999998, 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.840223955999999, 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:50:48.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.838190022999999, 2024-04-23T23:50:48.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:50:48.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.838178080999999, 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.128 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.836200947000002, 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:50:48.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.836172863000002, 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:50:48.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:50:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.836160327999998, 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:50:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:50:48.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:50:48.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.834219239999999, 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:50:48.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.150 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.834207893999999, 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.154 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.834197039999999, 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.155 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:50:48.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:48.159 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.834185749, 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:48.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:50:48.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.832146976000001, 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:50:48.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.832130993, 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:50:48.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:50:48.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.832110373999999, 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:50:48.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:50:48.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:50:48.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.832090376, 2024-04-23T23:50:48.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:50:48.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.181 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.832072096000001, 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:50:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.186 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.831069897999999, 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:50:48.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.831058399, 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.196 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:48.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:48.199 INFO:teuthology.orchestra.run.smithi092.stdout:4693 write 0x8c34aa7 thru 0x8c3dab0 (0x900a bytes) 2024-04-23T23:50:48.199 INFO:teuthology.orchestra.run.smithi092.stdout:4694 read 0x69b425 thru 0x69e9db (0x35b7 bytes) 2024-04-23T23:50:48.199 INFO:teuthology.orchestra.run.smithi092.stdout:4695 punch from 0x309975e to 0x309fd4d, (0x65ef bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4696 writesame 0x2fcb572 thru 0x2fd12a5 data_size 0xa(0x5d34 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4697 writesame 0xd742b19 thru 0xd744e68 data_size 0x8(0x2350 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4701 trunc from 0xd744e69 to 0x1185f4c 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4702 read 0x6d991f thru 0x6e5082 (0xb764 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4703 read 0xb2ad6e thru 0xb31bd1 (0x6e64 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4704 trunc from 0x1185f4c to 0x323d79f 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4706 write 0x4b2de36 thru 0x4b39012 (0xb1dd bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4708 read 0x3522270 thru 0x352a737 (0x84c8 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4711 read 0xe6cd07 thru 0xe7560c (0x8906 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4712 read 0x3b5f1c2 thru 0x3b6bde5 (0xcc24 bytes) 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.200 INFO:teuthology.orchestra.run.smithi092.stdout:4715 read 0x23aeac thru 0x248997 (0xdaec bytes) 2024-04-23T23:50:48.201 INFO:teuthology.orchestra.run.smithi092.stdout:4716 writesame 0x7ba966b thru 0x7bb7099 data_size 0x1(0xda2f bytes) 2024-04-23T23:50:48.201 INFO:teuthology.orchestra.run.smithi092.stdout:4717 write 0x549358e thru 0x549c7a9 (0x921c bytes) 2024-04-23T23:50:48.201 INFO:teuthology.orchestra.run.smithi092.stdout:4718 writesame 0xc187c91 thru 0xc18ef4d data_size 0x1(0x72bd bytes) 2024-04-23T23:50:48.201 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:48.211+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:48.201 INFO:teuthology.orchestra.run.smithi092.stdout:4719 write 0xdedfc7d thru 0xdee4e49 (0x51cd bytes) 2024-04-23T23:50:48.202 INFO:teuthology.orchestra.run.smithi092.stdout:4720 write 0x49e552f thru 0x49e89d9 (0x34ab bytes) 2024-04-23T23:50:48.202 INFO:teuthology.orchestra.run.smithi092.stdout:4721 read 0xcb34c62 thru 0xcb368c8 (0x1c67 bytes) 2024-04-23T23:50:48.202 INFO:teuthology.orchestra.run.smithi092.stdout:4723 write 0x72fa65 thru 0x73ab7e (0xb11a bytes) 2024-04-23T23:50:48.203 INFO:teuthology.orchestra.run.smithi092.stdout:4725 read 0xca4e98f thru 0xca596e5 (0xad57 bytes) 2024-04-23T23:50:48.204 INFO:teuthology.orchestra.run.smithi092.stdout:4727 write 0xc444b2b thru 0xc444f90 (0x466 bytes) 2024-04-23T23:50:48.205 INFO:teuthology.orchestra.run.smithi092.stdout:4728 read 0x6914157 thru 0x69156eb (0x1595 bytes) 2024-04-23T23:50:48.205 INFO:teuthology.orchestra.run.smithi092.stdout:4729 read 0x79ac7e3 thru 0x79b8635 (0xbe53 bytes) 2024-04-23T23:50:48.206 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.206 INFO:teuthology.orchestra.run.smithi092.stdout:4732 write 0x196d3e3 thru 0x197809f (0xacbd bytes) 2024-04-23T23:50:48.207 INFO:teuthology.orchestra.run.smithi092.stdout:4737 read 0x418e984 thru 0x4192da0 (0x441d bytes) 2024-04-23T23:50:48.208 INFO:teuthology.orchestra.run.smithi092.stdout:4739 writesame 0x8497f1b thru 0x849a85d data_size 0x7(0x2943 bytes) 2024-04-23T23:50:48.210 INFO:teuthology.orchestra.run.smithi092.stdout:4741 punch from 0xc328015 to 0xc329ad0, (0x1abb bytes) 2024-04-23T23:50:48.217 INFO:teuthology.orchestra.run.smithi092.stdout:4742 trunc from 0xdee4e4a to 0x7ea1018 2024-04-23T23:50:48.224 INFO:teuthology.orchestra.run.smithi092.stdout:4743 punch from 0x80f269 to 0x81309d, (0x3e34 bytes) 2024-04-23T23:50:48.224 INFO:teuthology.orchestra.run.smithi092.stdout:4744 read 0x4aa8182 thru 0x4aa8b5c (0x9db bytes) 2024-04-23T23:50:48.224 INFO:teuthology.orchestra.run.smithi092.stdout:4746 read 0x35389a thru 0x35a120 (0x6887 bytes) 2024-04-23T23:50:48.225 INFO:teuthology.orchestra.run.smithi092.stdout:4748 writesame 0x204a759 thru 0x204b7b3 data_size 0x1(0x105b bytes) 2024-04-23T23:50:48.232 INFO:teuthology.orchestra.run.smithi092.stdout:4751 trunc from 0x7ea1018 to 0xba182f8 2024-04-23T23:50:48.238 INFO:teuthology.orchestra.run.smithi092.stdout:4752 write 0x4ddbc9e thru 0x4ddf35b (0x36be bytes) 2024-04-23T23:50:48.239 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.245 INFO:teuthology.orchestra.run.smithi092.stdout:4754 write 0xc786de1 thru 0xc78ee1c (0x803c bytes) 2024-04-23T23:50:48.246 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.247 INFO:teuthology.orchestra.run.smithi092.stdout:4756 read 0x36ce976 thru 0x36d165d (0x2ce8 bytes) 2024-04-23T23:50:48.260 INFO:teuthology.orchestra.run.smithi092.stdout:4757 trunc from 0xc78ee1d to 0x1017114 2024-04-23T23:50:48.271 INFO:teuthology.orchestra.run.smithi092.stdout:4759 trunc from 0x1017114 to 0x19e356 2024-04-23T23:50:48.298 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:48.312+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:48.306 INFO:teuthology.orchestra.run.smithi092.stdout:4760 write 0x894b7db thru 0x895a940 (0xf166 bytes) 2024-04-23T23:50:48.308 INFO:teuthology.orchestra.run.smithi092.stdout:4761 read 0x4e2f7aa thru 0x4e38b5b (0x93b2 bytes) 2024-04-23T23:50:48.308 INFO:teuthology.orchestra.run.smithi092.stdout:4762 write 0x71bee43 thru 0x71c51ec (0x63aa bytes) 2024-04-23T23:50:48.309 INFO:teuthology.orchestra.run.smithi092.stdout:4763 trunc from 0x895a941 to 0x870c142 2024-04-23T23:50:48.315 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.315 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.315 INFO:teuthology.orchestra.run.smithi092.stdout:4768 read 0x691ccb4 thru 0x691f2a5 (0x25f2 bytes) 2024-04-23T23:50:48.322 INFO:teuthology.orchestra.run.smithi092.stdout:4769 trunc from 0x870c142 to 0xe0435f7 2024-04-23T23:50:48.327 INFO:teuthology.orchestra.run.smithi092.stdout:4770 read 0x25ddfaf thru 0x25e8427 (0xa479 bytes) 2024-04-23T23:50:48.328 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.328 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.342 INFO:teuthology.orchestra.run.smithi092.stdout:4774 trunc from 0xe0435f7 to 0x1becf5a 2024-04-23T23:50:48.346 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:48.347 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:48.349 INFO:teuthology.orchestra.run.smithi092.stdout:4775 punch from 0xf4ffbe to 0xf570d2, (0x7114 bytes) 2024-04-23T23:50:48.349 INFO:teuthology.orchestra.run.smithi092.stdout:4776 punch from 0x19351bf to 0x193e76c, (0x95ad bytes) 2024-04-23T23:50:48.349 INFO:teuthology.orchestra.run.smithi092.stdout:4777 read 0xde64db thru 0xde8dce (0x28f4 bytes) 2024-04-23T23:50:48.349 INFO:teuthology.orchestra.run.smithi092.stdout:4779 punch from 0x1a4ef2a to 0x1a516d4, (0x27aa bytes) 2024-04-23T23:50:48.350 INFO:teuthology.orchestra.run.smithi092.stdout:4780 punch from 0xdbbfba to 0xdc30ff, (0x7145 bytes) 2024-04-23T23:50:48.350 INFO:teuthology.orchestra.run.smithi092.stdout:4781 read 0x1b4d630 thru 0x1b56f7e (0x994f bytes) 2024-04-23T23:50:48.350 INFO:teuthology.orchestra.run.smithi092.stdout:4782 read 0x9c1cd8 thru 0x9c5a11 (0x3d3a bytes) 2024-04-23T23:50:48.350 INFO:teuthology.orchestra.run.smithi092.stdout:4783 read 0x16b5af3 thru 0x16bdaff (0x800d bytes) 2024-04-23T23:50:48.354 INFO:teuthology.orchestra.run.smithi092.stdout:4784 trunc from 0x1becf5a to 0x5d2be4e 2024-04-23T23:50:48.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:48.365 INFO:teuthology.orchestra.run.smithi092.stdout:4786 write 0x65cc1c6 thru 0x65d956e (0xd3a9 bytes) 2024-04-23T23:50:48.372 INFO:teuthology.orchestra.run.smithi092.stdout:4787 write 0x66b8fd7 thru 0x66c2937 (0x9961 bytes) 2024-04-23T23:50:48.374 INFO:teuthology.orchestra.run.smithi092.stdout:4788 write 0x564e971 thru 0x564f46c (0xafc bytes) 2024-04-23T23:50:48.384 INFO:teuthology.orchestra.run.smithi092.stdout:4789 write 0x77fe254 thru 0x7806ad1 (0x887e bytes) 2024-04-23T23:50:48.387 INFO:teuthology.orchestra.run.smithi092.stdout:4790 read 0x209773e thru 0x20a22a7 (0xab6a bytes) 2024-04-23T23:50:48.387 INFO:teuthology.orchestra.run.smithi092.stdout:4791 read 0x5d89926 thru 0x5d8b843 (0x1f1e bytes) 2024-04-23T23:50:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:4794 read 0xb1fb01 thru 0xb24e77 (0x5377 bytes) 2024-04-23T23:50:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:4797 read 0x7551629 thru 0x755e9ed (0xd3c5 bytes) 2024-04-23T23:50:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:4800 punch from 0x7287339 to 0x729521a, (0xdee1 bytes) 2024-04-23T23:50:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:4802 read 0x246b610 thru 0x247917b (0xdb6c bytes) 2024-04-23T23:50:48.397 INFO:teuthology.orchestra.run.smithi092.stdout:4804 write 0x993a44b thru 0x9947e99 (0xda4f bytes) 2024-04-23T23:50:48.399 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:48.413+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:48.406 INFO:teuthology.orchestra.run.smithi092.stdout:4805 write 0xb8cfec7 thru 0xb8db8d7 (0xba11 bytes) 2024-04-23T23:50:48.407 INFO:teuthology.orchestra.run.smithi092.stdout:4806 write 0x662b09c thru 0x6635408 (0xa36d bytes) 2024-04-23T23:50:48.409 INFO:teuthology.orchestra.run.smithi092.stdout:4808 write 0x3e33221 thru 0x3e34b72 (0x1952 bytes) 2024-04-23T23:50:48.410 INFO:teuthology.orchestra.run.smithi092.stdout:4809 punch from 0x8a28c3e to 0x8a31e4e, (0x9210 bytes) 2024-04-23T23:50:48.418 INFO:teuthology.orchestra.run.smithi092.stdout:4812 write 0xde25b24 thru 0xde3436c (0xe849 bytes) 2024-04-23T23:50:48.420 INFO:teuthology.orchestra.run.smithi092.stdout:4813 writesame 0x13543e0 thru 0x1361562 data_size 0x11(0xd183 bytes) 2024-04-23T23:50:48.425 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.425 INFO:teuthology.orchestra.run.smithi092.stdout:4815 write 0x1807eae thru 0x18099ce (0x1b21 bytes) 2024-04-23T23:50:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:4816 trunc from 0xde3436d to 0x60b4000 2024-04-23T23:50:48.442 INFO:teuthology.orchestra.run.smithi092.stdout:4817 punch from 0x150e73a to 0x151333d, (0x4c03 bytes) 2024-04-23T23:50:48.442 INFO:teuthology.orchestra.run.smithi092.stdout:4819 read 0x5314aab thru 0x5318a99 (0x3fef bytes) 2024-04-23T23:50:48.455 INFO:teuthology.orchestra.run.smithi092.stdout:4821 write 0xc88bd09 thru 0xc88f7cb (0x3ac3 bytes) 2024-04-23T23:50:48.457 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.457 INFO:teuthology.orchestra.run.smithi092.stdout:4824 read 0x313dfa7 thru 0x31427f5 (0x484f bytes) 2024-04-23T23:50:48.464 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:48.464 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:48.464 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:48.464 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:48.464 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:48.470 INFO:teuthology.orchestra.run.smithi092.stdout:4826 trunc from 0xc88f7cc to 0x1025c6c 2024-04-23T23:50:48.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:48.478 INFO:teuthology.orchestra.run.smithi092.stdout:4827 read 0x6b9306 thru 0x6bb50d (0x2208 bytes) 2024-04-23T23:50:48.478 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:48.478 INFO:teuthology.orchestra.run.smithi092.stdout:4830 punch from 0xeb56de to 0xec3765, (0xe087 bytes) 2024-04-23T23:50:48.490 INFO:teuthology.orchestra.run.smithi092.stdout:4831 write 0x645b15e thru 0x6464178 (0x901b bytes) 2024-04-23T23:50:48.497 INFO:teuthology.orchestra.run.smithi092.stdout:4832 trunc from 0x6464179 to 0x1ee18f6 2024-04-23T23:50:48.505 INFO:teuthology.orchestra.run.smithi092.stdout:4833 read 0xdcf228 thru 0xddde17 (0xebf0 bytes) 2024-04-23T23:50:48.514 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:48.528+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:48.517 INFO:teuthology.orchestra.run.smithi092.stdout:4834 write 0x7f4847a thru 0x7f4bb86 (0x370d bytes) 2024-04-23T23:50:48.519 INFO:teuthology.orchestra.run.smithi092.stdout:4835 write 0x2ea8036 thru 0x2eadbc0 (0x5b8b bytes) 2024-04-23T23:50:48.529 INFO:teuthology.orchestra.run.smithi092.stdout:4837 write 0xaef9d91 thru 0xaf03f8f (0xa1ff bytes) 2024-04-23T23:50:48.530 INFO:teuthology.orchestra.run.smithi092.stdout:4839 read 0x79e268d thru 0x79eddd3 (0xb747 bytes) 2024-04-23T23:50:48.530 INFO:teuthology.orchestra.run.smithi092.stdout:4840 punch from 0x1ff0c63 to 0x1ff1569, (0x906 bytes) 2024-04-23T23:50:48.537 INFO:teuthology.orchestra.run.smithi092.stdout:4841 write 0xbe9d066 thru 0xbea9529 (0xc4c4 bytes) 2024-04-23T23:50:48.541 INFO:teuthology.orchestra.run.smithi092.stdout:4842 write 0x2ada776 thru 0x2ae56d9 (0xaf64 bytes) 2024-04-23T23:50:48.547 INFO:teuthology.orchestra.run.smithi092.stdout:4843 trunc from 0xbea952a to 0x5fd2ddb 2024-04-23T23:50:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:4844 punch from 0x11aefde to 0x11b6026, (0x7048 bytes) 2024-04-23T23:50:48.563 INFO:teuthology.orchestra.run.smithi092.stdout:4845 write 0x953c631 thru 0x953fc81 (0x3651 bytes) 2024-04-23T23:50:48.565 INFO:teuthology.orchestra.run.smithi092.stdout:4846 punch from 0x20ff3d8 to 0x210bb7a, (0xc7a2 bytes) 2024-04-23T23:50:48.565 INFO:teuthology.orchestra.run.smithi092.stdout:4847 trunc from 0x953fc82 to 0x9f2ec96 2024-04-23T23:50:48.571 INFO:teuthology.orchestra.run.smithi092.stdout:4848 writesame 0x5bd94fd thru 0x5be7496 data_size 0x1(0xdf9a bytes) 2024-04-23T23:50:48.600 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:48.614+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:48.604 INFO:teuthology.orchestra.run.smithi092.stdout:4852 trunc from 0x9f2ec96 to 0xe3897af 2024-04-23T23:50:48.609 INFO:teuthology.orchestra.run.smithi092.stdout:4853 read 0xa786cb thru 0xa7bccf (0x3605 bytes) 2024-04-23T23:50:48.611 INFO:teuthology.orchestra.run.smithi092.stdout:4854 write 0xcfc86c7 thru 0xcfd83cc (0xfd06 bytes) 2024-04-23T23:50:48.612 INFO:teuthology.orchestra.run.smithi092.stdout:4855 read 0x1c3911c thru 0x1c3f3b5 (0x629a bytes) 2024-04-23T23:50:48.613 INFO:teuthology.orchestra.run.smithi092.stdout:4857 punch from 0xaf68e00 to 0xaf6dbef, (0x4def bytes) 2024-04-23T23:50:48.613 INFO:teuthology.orchestra.run.smithi092.stdout:4858 clone 31 order 21 su 1048576 sc 9 2024-04-23T23:50:48.620 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.114632676999999, 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.626 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.114599629000001, 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:50:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:50:48.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.114577130000001, 2024-04-23T23:50:48.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:50:48.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:50:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.112707213, 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.636 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.112678737, 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.641 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.112656293000001, 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:50:48.645 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:50:48.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:50:48.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.110700564999998, 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:50:48.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:50:48.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.110681438, 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:50:48.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:50:48.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.11065529, 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:48.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:50:48.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.110634598000001, 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:50:48.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.110612872000001, 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.668 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:50:48.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.108715982, 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.672 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:48.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:50:48.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.108691534999998, 2024-04-23T23:50:48.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.108674441000002, 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:48.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.106697059999998, 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.689 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.106678182, 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:50:48.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.104697141999999, 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:50:48.694 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.698 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.104657339999999, 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.104635316, 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:50:48.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:48.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.104613775000001, 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:50:48.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:48.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:48.712 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:48.712 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:48.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:48.713 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:48.727+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:48.814 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:48.828+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:48.855 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:48.856 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:48.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:48.915 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:48.929+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:49.006 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:49.006 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:49.007 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:49.014 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:49.016 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:49.030+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:49.116 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:49.130+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:49.117 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.929835210999997, 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.581098543000003, 2024-04-23T23:50:49.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:49.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:49.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.581082090000002, 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:49.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.581055349000003, 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:49.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:49.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.581036554000001, 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:49.137 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:49.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:49.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.581018970000002, 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:49.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:49.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.569323466, 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:49.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:49.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:49.150 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.569147778999998, 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.154 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.568996259999999, 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.568713633999998, 2024-04-23T23:50:49.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.162 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.568567768999998, 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:49.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.567618084999999, 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:49.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:49.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.567595191000002, 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.171 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:49.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:49.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.551579816, 2024-04-23T23:50:49.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:49.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.179 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.551570181999999, 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.536537821000003, 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:49.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.536516497000001, 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:49.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.536493671999999, 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.536470993000002, 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.53416927, 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:49.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:49.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:49.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:49.205 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:49.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:49.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:50:49.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:49.252 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:49.266+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:49.255 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone30 intact 2024-04-23T23:50:49.318 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:49.332+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:49.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:49.327 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:49.327 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:49.327 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:49.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:49.343 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:49.343 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:49.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:49.366 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:49.385 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:49.385 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:49.402 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:49.419 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:49.433+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:49.420 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:49.420 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:49.454 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:49.455 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:49.455 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:49.458 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-04-23T23:50:49.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:49.473 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:49.473 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:49.476 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:50:49.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T23:50:49.492 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:49.492 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:49.555 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:49.569+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:49.621 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:49.636+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:49.640 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:49.640 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:49.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:49.723 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:49.737+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:49.767 INFO:teuthology.orchestra.run.smithi092.stdout:4860 punch from 0x83505cf to 0x8360398, (0xfdc9 bytes) 2024-04-23T23:50:49.770 INFO:teuthology.orchestra.run.smithi092.stdout:4861 punch from 0x20b75f2 to 0x20b8c2e, (0x163c bytes) 2024-04-23T23:50:49.770 INFO:teuthology.orchestra.run.smithi092.stdout:4862 write 0x765a78c thru 0x7666a54 (0xc2c9 bytes) 2024-04-23T23:50:49.774 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.775 INFO:teuthology.orchestra.run.smithi092.stdout:4868 write 0x4b849b3 thru 0x4b92d81 (0xe3cf bytes) 2024-04-23T23:50:49.786 INFO:teuthology.orchestra.run.smithi092.stdout:4869 trunc from 0xe3897af to 0x7560325 2024-04-23T23:50:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:49.803 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.803 INFO:teuthology.orchestra.run.smithi092.stdout:4871 read 0x27b4279 thru 0x27baaa4 (0x682c bytes) 2024-04-23T23:50:49.804 INFO:teuthology.orchestra.run.smithi092.stdout:4875 read 0x26b7091 thru 0x26bda0c (0x697c bytes) 2024-04-23T23:50:49.804 INFO:teuthology.orchestra.run.smithi092.stdout:4876 trunc from 0x7560325 to 0x7679910 2024-04-23T23:50:49.810 INFO:teuthology.orchestra.run.smithi092.stdout:4878 writesame 0x864e8e thru 0x872376 data_size 0xb(0xd4e9 bytes) 2024-04-23T23:50:49.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:49.822 INFO:teuthology.orchestra.run.smithi092.stdout:4881 writesame 0x8422d36 thru 0x8425361 data_size 0xe(0x262c bytes) 2024-04-23T23:50:49.822 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.822 INFO:teuthology.orchestra.run.smithi092.stdout:4888 read 0x63378fe thru 0x633bfa8 (0x46ab bytes) 2024-04-23T23:50:49.823 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:49.837+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:49.831 INFO:teuthology.orchestra.run.smithi092.stdout:4889 trunc from 0x8425362 to 0xe823939 2024-04-23T23:50:49.837 INFO:teuthology.orchestra.run.smithi092.stdout:4890 punch from 0x9997e5b to 0x99a1e9f, (0xa044 bytes) 2024-04-23T23:50:49.837 INFO:teuthology.orchestra.run.smithi092.stdout:4891 read 0xe4f17bf thru 0xe4fb4ff (0x9d41 bytes) 2024-04-23T23:50:49.837 INFO:teuthology.orchestra.run.smithi092.stdout:4892 punch from 0x799c5d6 to 0x79a468c, (0x80b6 bytes) 2024-04-23T23:50:49.837 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.837 INFO:teuthology.orchestra.run.smithi092.stdout:4895 writesame 0xd170996 thru 0xd17ef5c data_size 0x1(0xe5c7 bytes) 2024-04-23T23:50:49.868 INFO:teuthology.orchestra.run.smithi092.stdout:4896 write 0x487e54f thru 0x487e76f (0x221 bytes) 2024-04-23T23:50:49.873 INFO:teuthology.orchestra.run.smithi092.stdout:4897 write 0xda13f3e thru 0xda2133f (0xd402 bytes) 2024-04-23T23:50:49.876 INFO:teuthology.orchestra.run.smithi092.stdout:4898 write 0x28e4787 thru 0x28f1626 (0xcea0 bytes) 2024-04-23T23:50:49.880 INFO:teuthology.orchestra.run.smithi092.stdout:4899 read 0xb122479 thru 0xb129097 (0x6c1f bytes) 2024-04-23T23:50:49.896 INFO:teuthology.orchestra.run.smithi092.stdout:4900 trunc from 0xe823939 to 0xd53bc0 2024-04-23T23:50:49.920 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.920 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.924 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:49.938+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:49.925 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:49.925 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:49.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:49.941 INFO:teuthology.orchestra.run.smithi092.stdout:4903 writesame 0xdb34fc3 thru 0xdb3b06e data_size 0x2e(0x60ac bytes) 2024-04-23T23:50:49.941 INFO:teuthology.orchestra.run.smithi092.stdout:4905 writesame 0xb0f07d6 thru 0xb0f8e09 data_size 0x1c(0x8634 bytes) 2024-04-23T23:50:49.946 INFO:teuthology.orchestra.run.smithi092.stdout:4906 write 0x74c7521 thru 0x74c9c0e (0x26ee bytes) 2024-04-23T23:50:49.949 INFO:teuthology.orchestra.run.smithi092.stdout:4907 write 0xc56b16a thru 0xc579aaa (0xe941 bytes) 2024-04-23T23:50:49.951 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:49.961 INFO:teuthology.orchestra.run.smithi092.stdout:4909 trunc from 0xdb3b06f to 0x56542c9 2024-04-23T23:50:49.974 INFO:teuthology.orchestra.run.smithi092.stdout:4910 writesame 0x34e3730 thru 0x34f22cc data_size 0x1(0xeb9d bytes) 2024-04-23T23:50:50.007 INFO:teuthology.orchestra.run.smithi092.stdout:4912 read 0x1cf1cf1 thru 0x1d01307 (0xf617 bytes) 2024-04-23T23:50:50.008 INFO:teuthology.orchestra.run.smithi092.stdout:4913 writesame 0x2712c15 thru 0x2717802 data_size 0x1(0x4bee bytes) 2024-04-23T23:50:50.025 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:50.039+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:50.036 INFO:teuthology.orchestra.run.smithi092.stdout:4914 write 0xcf1c402 thru 0xcf24128 (0x7d27 bytes) 2024-04-23T23:50:50.039 INFO:teuthology.orchestra.run.smithi092.stdout:4918 read 0x6899d89 thru 0x68a8960 (0xebd8 bytes) 2024-04-23T23:50:50.039 INFO:teuthology.orchestra.run.smithi092.stdout:4919 write 0xa2b393e thru 0xa2c1075 (0xd738 bytes) 2024-04-23T23:50:50.045 INFO:teuthology.orchestra.run.smithi092.stdout:4920 trunc from 0xcf24129 to 0xa9a20ea 2024-04-23T23:50:50.050 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.051 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.058 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:50.063 INFO:teuthology.orchestra.run.smithi092.stdout:4922 trunc from 0xa9a20ea to 0x62f76c1 2024-04-23T23:50:50.083 INFO:teuthology.orchestra.run.smithi092.stdout:4923 trunc from 0x62f76c1 to 0xe99d672 2024-04-23T23:50:50.088 INFO:teuthology.orchestra.run.smithi092.stdout:4924 punch from 0xe42ed8b to 0xe434846, (0x5abb bytes) 2024-04-23T23:50:50.088 INFO:teuthology.orchestra.run.smithi092.stdout:4925 write 0x77613db thru 0x7763ecd (0x2af3 bytes) 2024-04-23T23:50:50.091 INFO:teuthology.orchestra.run.smithi092.stdout:4927 read 0xcc2472 thru 0xcc72d5 (0x4e64 bytes) 2024-04-23T23:50:50.103 INFO:teuthology.orchestra.run.smithi092.stdout:4928 trunc from 0xe99d672 to 0x65e4ac9 2024-04-23T23:50:50.122 INFO:teuthology.orchestra.run.smithi092.stdout:4929 write 0x9a0f75b thru 0x9a1006a (0x910 bytes) 2024-04-23T23:50:50.124 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.124 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.125 INFO:teuthology.orchestra.run.smithi092.stdout:4935 punch from 0x4697936 to 0x469a095, (0x275f bytes) 2024-04-23T23:50:50.125 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:50.139+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:50.126 INFO:teuthology.orchestra.run.smithi092.stdout:4936 trunc from 0x9a1006b to 0x82db37b 2024-04-23T23:50:50.136 INFO:teuthology.orchestra.run.smithi092.stdout:4937 read 0x3811b47 thru 0x3818e14 (0x72ce bytes) 2024-04-23T23:50:50.136 INFO:teuthology.orchestra.run.smithi092.stdout:4938 write 0x89f5d5 thru 0x8a35af (0x3fdb bytes) 2024-04-23T23:50:50.149 INFO:teuthology.orchestra.run.smithi092.stdout:4939 writesame 0xc0a1a20 thru 0xc0a6e42 data_size 0x11(0x5423 bytes) 2024-04-23T23:50:50.149 INFO:teuthology.orchestra.run.smithi092.stdout:4940 writesame 0xbc6a469 thru 0xbc71e0b data_size 0x1(0x79a3 bytes) 2024-04-23T23:50:50.165 INFO:teuthology.orchestra.run.smithi092.stdout:4942 write 0x473468e thru 0x4739e75 (0x57e8 bytes) 2024-04-23T23:50:50.170 INFO:teuthology.orchestra.run.smithi092.stdout:4943 writesame 0x5d6011f thru 0x5d6d4e9 data_size 0x5d(0xd3cb bytes) 2024-04-23T23:50:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:4944 punch from 0x270203c to 0x2710e96, (0xee5a bytes) 2024-04-23T23:50:50.185 INFO:teuthology.orchestra.run.smithi092.stdout:4945 trunc from 0xc0a6e43 to 0x41fa7a2 2024-04-23T23:50:50.190 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T23:50:50.193 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:50:50.198 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.198 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.199 INFO:teuthology.orchestra.run.smithi092.stdout:4948 write 0x41e2866 thru 0x41e3b8e (0x1329 bytes) 2024-04-23T23:50:50.205 INFO:teuthology.orchestra.run.smithi092.stdout:4949 read 0x327bd0f thru 0x328190d (0x5bff bytes) 2024-04-23T23:50:50.205 INFO:teuthology.orchestra.run.smithi092.stdout:4950 read 0x50339a thru 0x50c1f8 (0x8e5f bytes) 2024-04-23T23:50:50.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T23:50:50.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:50.214 INFO:teuthology.orchestra.run.smithi092.stdout:4952 write 0x546bc6f thru 0x547b944 (0xfcd6 bytes) 2024-04-23T23:50:50.217 INFO:teuthology.orchestra.run.smithi092.stdout:4954 read 0x14d335d thru 0x14d3748 (0x3ec bytes) 2024-04-23T23:50:50.226 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:50.240+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:50.231 INFO:teuthology.orchestra.run.smithi092.stdout:4957 write 0xd4cdae7 thru 0xd4d9491 (0xb9ab bytes) 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:4959 punch from 0x33795f7 to 0x337b425, (0x1e2e bytes) 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:4961 read 0x5fd7e6a thru 0x5fdb02e (0x31c5 bytes) 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:4962 punch from 0x432e602 to 0x433281d, (0x421b bytes) 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:4964 read 0xb1dbce6 thru 0xb1dd7a0 (0x1abb bytes) 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:4970 read 0xa4249c3 thru 0xa42896b (0x3fa9 bytes) 2024-04-23T23:50:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:4971 punch from 0xa8f71aa to 0xa8fbe31, (0x4c87 bytes) 2024-04-23T23:50:50.239 INFO:teuthology.orchestra.run.smithi092.stdout:4972 writesame 0xd4d3574 thru 0xd4dddad data_size 0x3d(0xa83a bytes) 2024-04-23T23:50:50.242 INFO:teuthology.orchestra.run.smithi092.stdout:4973 writesame 0x3aae566 thru 0x3aaf2ed data_size 0x4(0xd88 bytes) 2024-04-23T23:50:50.245 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.245 INFO:teuthology.orchestra.run.smithi092.stdout:4975 writesame 0x6c567c9 thru 0x6c58c88 data_size 0x38(0x24c0 bytes) 2024-04-23T23:50:50.248 INFO:teuthology.orchestra.run.smithi092.stdout:4976 read 0xcc30779 thru 0xcc3bc2f (0xb4b7 bytes) 2024-04-23T23:50:50.248 INFO:teuthology.orchestra.run.smithi092.stdout:4977 write 0xaef5065 thru 0xaef999e (0x493a bytes) 2024-04-23T23:50:50.253 INFO:teuthology.orchestra.run.smithi092.stdout:4979 read 0x2c9a911 thru 0x2c9ae67 (0x557 bytes) 2024-04-23T23:50:50.253 INFO:teuthology.orchestra.run.smithi092.stdout:4980 read 0x2f53dcd thru 0x2f599d0 (0x5c04 bytes) 2024-04-23T23:50:50.253 INFO:teuthology.orchestra.run.smithi092.stdout:4981 read 0xc5c2c6a thru 0xc5c93c5 (0x675c bytes) 2024-04-23T23:50:50.253 INFO:teuthology.orchestra.run.smithi092.stdout:4983 writesame 0x46f582b thru 0x4702aa6 data_size 0x26(0xd27c bytes) 2024-04-23T23:50:50.256 INFO:teuthology.orchestra.run.smithi092.stdout:4984 read 0x54e091 thru 0x54ea98 (0xa08 bytes) 2024-04-23T23:50:50.257 INFO:teuthology.orchestra.run.smithi092.stdout:4985 trunc from 0xd4dddae to 0xcf4ecb3 2024-04-23T23:50:50.270 INFO:teuthology.orchestra.run.smithi092.stdout:4987 writesame 0xe0318be thru 0xe034ea6 data_size 0x1(0x35e9 bytes) 2024-04-23T23:50:50.283 INFO:teuthology.orchestra.run.smithi092.stdout:4988 trunc from 0xe034ea7 to 0x36b2f1b 2024-04-23T23:50:50.314 INFO:teuthology.orchestra.run.smithi092.stdout:4989 write 0x90c889d thru 0x90d2d88 (0xa4ec bytes) 2024-04-23T23:50:50.317 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.317 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.317 INFO:teuthology.orchestra.run.smithi092.stdout:4995 write 0x58c4701 thru 0x58cf5f0 (0xaef0 bytes) 2024-04-23T23:50:50.327 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:50.341+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:50.329 INFO:teuthology.orchestra.run.smithi092.stdout:4996 write 0xb014b86 thru 0xb01f872 (0xaced bytes) 2024-04-23T23:50:50.330 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.330 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.333 INFO:teuthology.orchestra.run.smithi092.stdout:4997 read 0x50f8195 thru 0x50fcacc (0x4938 bytes) 2024-04-23T23:50:50.333 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.333 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:50.334 INFO:teuthology.orchestra.run.smithi092.stdout:5002 read 0x58fc965 thru 0x590518c (0x8828 bytes) 2024-04-23T23:50:50.334 INFO:teuthology.orchestra.run.smithi092.stdout:5004 read 0x27223e6 thru 0x2731c8b (0xf8a6 bytes) 2024-04-23T23:50:50.334 INFO:teuthology.orchestra.run.smithi092.stdout:5005 punch from 0x794e153 to 0x795ba5d, (0xd90a bytes) 2024-04-23T23:50:50.334 INFO:teuthology.orchestra.run.smithi092.stdout:5006 write 0x729208e thru 0x7298ea6 (0x6e19 bytes) 2024-04-23T23:50:50.338 INFO:teuthology.orchestra.run.smithi092.stdout:5007 write 0x34d6c6c thru 0x34d9e8c (0x3221 bytes) 2024-04-23T23:50:50.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:50.341 INFO:teuthology.orchestra.run.smithi092.stdout:5008 read 0x7d97f6d thru 0x7da4e25 (0xceb9 bytes) 2024-04-23T23:50:50.341 INFO:teuthology.orchestra.run.smithi092.stdout:5010 punch from 0x8d77805 to 0x8d7ea88, (0x7283 bytes) 2024-04-23T23:50:50.341 INFO:teuthology.orchestra.run.smithi092.stdout:5012 read 0x2a82ac thru 0x2b6c3b (0xe990 bytes) 2024-04-23T23:50:50.341 INFO:teuthology.orchestra.run.smithi092.stdout:5013 write 0x481eefe thru 0x481f277 (0x37a bytes) 2024-04-23T23:50:50.345 INFO:teuthology.orchestra.run.smithi092.stdout:5015 read 0xaa3a0e6 thru 0xaa3dcfe (0x3c19 bytes) 2024-04-23T23:50:50.349 INFO:teuthology.orchestra.run.smithi092.stdout:5017 trunc from 0xb01f873 to 0xe6d56ca 2024-04-23T23:50:50.354 INFO:teuthology.orchestra.run.smithi092.stdout:5018 clone 32 order 20 su 16384 sc 15 2024-04-23T23:50:50.429 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:50.443+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:50.444 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.444 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:50.530 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:50.543+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:50.596 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.598 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:50.650 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:50.664+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:50.708 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.708 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:50.731 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:50.745+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:50.832 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:50.846+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:50.858 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.858 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:50.932 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:50.946+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:50.971 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:50.971 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:50.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:51.033 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:51.046+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:51.123 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:51.123 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:51.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:51.133 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:51.147+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:51.198 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T23:50:51.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T23:50:51.234 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:51.248+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:51.239 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:51.239 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:51.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:51.267 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone31 from 0xe6d56ca (overlap 0xd53bc0) to 0x3ff853 2024-04-23T23:50:51.335 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:51.349+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:51.351 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-04-23T23:50:51.395 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:51.395 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:51.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:51.446 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:51.460+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:51.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:51.537 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:51.551+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:51.547 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:51.547 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:51.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:51.640 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:51.654+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:51.641 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:51.647 INFO:teuthology.orchestra.run.smithi092.stdout:5019 writesame 0x6e0e809 thru 0x6e13db5 data_size 0x3(0x55ad bytes) 2024-04-23T23:50:51.658 INFO:teuthology.orchestra.run.smithi092.stdout:5020 writesame 0x142d62 thru 0x15005a data_size 0x33(0xd2f9 bytes) 2024-04-23T23:50:51.663 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:51.682 INFO:teuthology.orchestra.run.smithi092.stdout:5022 writesame 0xecd0810 thru 0xecde199 data_size 0x5(0xd98a bytes) 2024-04-23T23:50:51.686 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:51.688 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.688 INFO:teuthology.orchestra.run.smithi092.stdout:5024 write 0xb171dd5 thru 0xb17e0f3 (0xc31f bytes) 2024-04-23T23:50:51.699 INFO:teuthology.orchestra.run.smithi092.stdout:5025 punch from 0x13b0b6e to 0x13b696c, (0x5dfe bytes) 2024-04-23T23:50:51.699 INFO:teuthology.orchestra.run.smithi092.stdout:5026 writesame 0xb355e3b thru 0xb35c1d1 data_size 0x1(0x6397 bytes) 2024-04-23T23:50:51.707 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:51.707 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:51.708 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:51.709 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:51.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:51.718 INFO:teuthology.orchestra.run.smithi092.stdout:5027 write 0xd3adcfd thru 0xd3b9afb (0xbdff bytes) 2024-04-23T23:50:51.724 INFO:teuthology.orchestra.run.smithi092.stdout:5030 punch from 0xb9a64b1 to 0xb9ad956, (0x74a5 bytes) 2024-04-23T23:50:51.724 INFO:teuthology.orchestra.run.smithi092.stdout:5031 write 0xad8a065 thru 0xad93a2b (0x99c7 bytes) 2024-04-23T23:50:51.729 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:51.729 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:51.732 INFO:teuthology.orchestra.run.smithi092.stdout:5032 read 0x844798f thru 0x844ada1 (0x3413 bytes) 2024-04-23T23:50:51.732 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.732 INFO:teuthology.orchestra.run.smithi092.stdout:5034 write 0x682894f thru 0x683068c (0x7d3e bytes) 2024-04-23T23:50:51.738 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:51.751+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:51.739 INFO:teuthology.orchestra.run.smithi092.stdout:5035 write 0xdc5a4a9 thru 0xdc60041 (0x5b99 bytes) 2024-04-23T23:50:51.741 INFO:teuthology.orchestra.run.smithi092.stdout:5036 read 0x7aa7dc0 thru 0x7ab0197 (0x83d8 bytes) 2024-04-23T23:50:51.741 INFO:teuthology.orchestra.run.smithi092.stdout:5037 write 0x991d961 thru 0x9926fc9 (0x9669 bytes) 2024-04-23T23:50:51.750 INFO:teuthology.orchestra.run.smithi092.stdout:5038 read 0x28097e0 thru 0x281723b (0xda5c bytes) 2024-04-23T23:50:51.750 INFO:teuthology.orchestra.run.smithi092.stdout:5040 writesame 0x87d0225 thru 0x87d166d data_size 0x3(0x1449 bytes) 2024-04-23T23:50:51.751 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:51.755 INFO:teuthology.orchestra.run.smithi092.stdout:5041 write 0x936625e thru 0x936e7d1 (0x8574 bytes) 2024-04-23T23:50:51.764 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.764 INFO:teuthology.orchestra.run.smithi092.stdout:5043 read 0x11a424 thru 0x128866 (0xe443 bytes) 2024-04-23T23:50:51.765 INFO:teuthology.orchestra.run.smithi092.stdout:5044 write 0xe998c2f thru 0xe99a743 (0x1b15 bytes) 2024-04-23T23:50:51.768 INFO:teuthology.orchestra.run.smithi092.stdout:5045 write 0xd674d89 thru 0xd678bcd (0x3e45 bytes) 2024-04-23T23:50:51.772 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:51.772 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:51.775 INFO:teuthology.orchestra.run.smithi092.stdout:5047 read 0xcb98b16 thru 0xcb99c7a (0x1165 bytes) 2024-04-23T23:50:51.775 INFO:teuthology.orchestra.run.smithi092.stdout:5049 read 0x84133 thru 0x8d38b (0x9259 bytes) 2024-04-23T23:50:51.783 INFO:teuthology.orchestra.run.smithi092.stdout:5050 trunc from 0xecde19a to 0x8ac9b2d 2024-04-23T23:50:51.792 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:51.793 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:51.818 INFO:teuthology.orchestra.run.smithi092.stdout:5053 punch from 0x69c4b2b to 0x69ca8fd, (0x5dd2 bytes) 2024-04-23T23:50:51.818 INFO:teuthology.orchestra.run.smithi092.stdout:5054 punch from 0x2010b7d to 0x2010f26, (0x3a9 bytes) 2024-04-23T23:50:51.828 INFO:teuthology.orchestra.run.smithi092.stdout:5055 write 0xb4e6075 thru 0xb4e94a0 (0x342c bytes) 2024-04-23T23:50:51.833 INFO:teuthology.orchestra.run.smithi092.stdout:5057 trunc from 0xb4e94a1 to 0xc791de9 2024-04-23T23:50:51.839 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:51.852+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:51.839 INFO:teuthology.orchestra.run.smithi092.stdout:5058 read 0x1cbb376 thru 0x1cc4d0d (0x9998 bytes) 2024-04-23T23:50:51.840 INFO:teuthology.orchestra.run.smithi092.stdout:5059 write 0x361ebf6 thru 0x3620e0a (0x2215 bytes) 2024-04-23T23:50:51.845 INFO:teuthology.orchestra.run.smithi092.stdout:5060 write 0x6594a12 thru 0x659a59c (0x5b8b bytes) 2024-04-23T23:50:51.847 INFO:teuthology.orchestra.run.smithi092.stdout:5061 read 0xc2396b1 thru 0xc241e1a (0x876a bytes) 2024-04-23T23:50:51.847 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.848 INFO:teuthology.orchestra.run.smithi092.stdout:5063 trunc from 0xc791de9 to 0xb5b1149 2024-04-23T23:50:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:5066 writesame 0x1adfb98 thru 0x1ae9438 data_size 0x1(0x98a1 bytes) 2024-04-23T23:50:51.867 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:51.867 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:51.867 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:51.867 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:51.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:51.881 INFO:teuthology.orchestra.run.smithi092.stdout:5067 writesame 0x4cebe9d thru 0x4cedf00 data_size 0x6(0x2064 bytes) 2024-04-23T23:50:51.887 INFO:teuthology.orchestra.run.smithi092.stdout:5069 read 0x4a1c65e thru 0x4a21441 (0x4de4 bytes) 2024-04-23T23:50:51.896 INFO:teuthology.orchestra.run.smithi092.stdout:5071 trunc from 0xb5b1149 to 0x4812bef 2024-04-23T23:50:51.931 INFO:teuthology.orchestra.run.smithi092.stdout:5073 writesame 0x2e5df73 thru 0x2e6b20a data_size 0x8(0xd298 bytes) 2024-04-23T23:50:51.940 INFO:teuthology.orchestra.run.smithi092.stdout:5074 punch from 0x17d9c54 to 0x17da751, (0xafd bytes) 2024-04-23T23:50:51.940 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:51.953+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:51.954 INFO:teuthology.orchestra.run.smithi092.stdout:5075 write 0xb466cd2 thru 0xb46a4a9 (0x37d8 bytes) 2024-04-23T23:50:51.960 INFO:teuthology.orchestra.run.smithi092.stdout:5078 punch from 0x2250bbf to 0x22608c1, (0xfd02 bytes) 2024-04-23T23:50:51.961 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.961 INFO:teuthology.orchestra.run.smithi092.stdout:5081 read 0x12f8a25 thru 0x12fd8e4 (0x4ec0 bytes) 2024-04-23T23:50:51.961 INFO:teuthology.orchestra.run.smithi092.stdout:5082 punch from 0x5b356 to 0x5c0ec, (0xd96 bytes) 2024-04-23T23:50:51.961 INFO:teuthology.orchestra.run.smithi092.stdout:5083 punch from 0x97d5b4 to 0x985b4b, (0x8597 bytes) 2024-04-23T23:50:51.961 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.961 INFO:teuthology.orchestra.run.smithi092.stdout:5088 write 0x5e73f3d thru 0x5e7b945 (0x7a09 bytes) 2024-04-23T23:50:51.968 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.968 INFO:teuthology.orchestra.run.smithi092.stdout:5090 write 0xa22e5eb thru 0xa235904 (0x731a bytes) 2024-04-23T23:50:51.978 INFO:teuthology.orchestra.run.smithi092.stdout:5091 read 0x4248a94 thru 0x42543be (0xb92b bytes) 2024-04-23T23:50:51.978 INFO:teuthology.orchestra.run.smithi092.stdout:5092 punch from 0x83a7524 to 0x83ad399, (0x5e75 bytes) 2024-04-23T23:50:51.978 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:51.986 INFO:teuthology.orchestra.run.smithi092.stdout:5094 trunc from 0xb46a4aa to 0x47acf86 2024-04-23T23:50:51.987 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:51.987 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:51.987 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:51.987 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:51.987 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:51.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:52.014 INFO:teuthology.orchestra.run.smithi092.stdout:5095 write 0x8151cc2 thru 0x815f750 (0xda8f bytes) 2024-04-23T23:50:52.021 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.021 INFO:teuthology.orchestra.run.smithi092.stdout:5098 write 0x5a66ef5 thru 0x5a71948 (0xaa54 bytes) 2024-04-23T23:50:52.034 INFO:teuthology.orchestra.run.smithi092.stdout:5099 write 0xaccd431 thru 0xacd6141 (0x8d11 bytes) 2024-04-23T23:50:52.039 INFO:teuthology.orchestra.run.smithi092.stdout:5100 write 0x8b3f7e6 thru 0x8b3f90d (0x128 bytes) 2024-04-23T23:50:52.041 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:52.054+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:52.042 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.042 INFO:teuthology.orchestra.run.smithi092.stdout:5102 punch from 0xa5be4f4 to 0xa5c6ca3, (0x87af bytes) 2024-04-23T23:50:52.042 INFO:teuthology.orchestra.run.smithi092.stdout:5104 punch from 0x1d9f267 to 0x1da4df4, (0x5b8d bytes) 2024-04-23T23:50:52.042 INFO:teuthology.orchestra.run.smithi092.stdout:5105 writesame 0x8cc05c7 thru 0x8cc1494 data_size 0x5(0xece bytes) 2024-04-23T23:50:52.044 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:5109 trunc from 0xacd6142 to 0x453e2d5 2024-04-23T23:50:52.068 INFO:teuthology.orchestra.run.smithi092.stdout:5110 read 0x7c5cac thru 0x7cef47 (0x929c bytes) 2024-04-23T23:50:52.069 INFO:teuthology.orchestra.run.smithi092.stdout:5112 flatten 2024-04-23T23:50:52.127 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:52.127 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:52.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:52.127 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:52.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.127 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.620857285, 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:52.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.132 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.620824237000001, 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:50:52.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:50:52.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.620801738000001, 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.618931821, 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:50:52.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:50:52.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.618903345, 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:52.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:50:52.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:50:52.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.618880901000001, 2024-04-23T23:50:52.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:50:52.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.616925172999998, 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:50:52.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.159 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.616906046, 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:50:52.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.616879898000001, 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:50:52.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:52.168 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.616859206000001, 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:52.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.616837480000001, 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.173 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.61494059, 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:52.181 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.614916142999999, 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:50:52.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.614899049000002, 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:50:52.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.612921667999998, 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.190 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:50:52.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:50:52.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:52.194 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.61290279, 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:50:52.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:50:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.198 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.610921749999999, 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:50:52.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.610881947999999, 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.203 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.207 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.610859924, 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:50:52.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.610838383000001, 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:50:52.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.216 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:52.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:50:52.219 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:52.220 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:52.232+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:5115 write 0xc196c5a thru 0xc196cbf (0x66 bytes) 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:5117 trunc from 0xc196cc0 to 0x56badcc 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:5119 trunc from 0x56badcc to 0x743c0cb 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:5121 writesame 0xc942bba thru 0xc951e21 data_size 0x4(0xf268 bytes) 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:5122 write 0x60d24bd thru 0x60def15 (0xca59 bytes) 2024-04-23T23:50:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:5123 write 0x90a3f5e thru 0x90aef1b (0xafbe bytes) 2024-04-23T23:50:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T23:50:52.221 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.221 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.221 INFO:teuthology.orchestra.run.smithi092.stdout:5126 write 0xc03d403 thru 0xc041390 (0x3f8e bytes) 2024-04-23T23:50:52.221 INFO:teuthology.orchestra.run.smithi092.stdout:5127 read 0x2ec5220 thru 0x2ecce19 (0x7bfa bytes) 2024-04-23T23:50:52.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T23:50:52.223 INFO:teuthology.orchestra.run.smithi092.stdout:5128 trunc from 0xc951e22 to 0x61aa5bf 2024-04-23T23:50:52.248 INFO:teuthology.orchestra.run.smithi092.stdout:5130 writesame 0xd4571f8 thru 0xd460160 data_size 0x1(0x8f69 bytes) 2024-04-23T23:50:52.260 INFO:teuthology.orchestra.run.smithi092.stdout:5132 trunc from 0xd460161 to 0x36a3a52 2024-04-23T23:50:52.281 INFO:teuthology.orchestra.run.smithi092.stdout:5133 read 0x20a5e33 thru 0x20a9c99 (0x3e67 bytes) 2024-04-23T23:50:52.291 INFO:teuthology.orchestra.run.smithi092.stdout:5134 write 0x6cd717b thru 0x6cda062 (0x2ee8 bytes) 2024-04-23T23:50:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:5135 read 0x3328972 thru 0x332b038 (0x26c7 bytes) 2024-04-23T23:50:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:5136 read 0xbd8b21 thru 0xbdc03d (0x351d bytes) 2024-04-23T23:50:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:5137 read 0x687e4f1 thru 0x688d6e3 (0xf1f3 bytes) 2024-04-23T23:50:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:5139 read 0x2f684dd thru 0x2f700dc (0x7c00 bytes) 2024-04-23T23:50:52.306 INFO:teuthology.orchestra.run.smithi092.stdout:5140 write 0x74f1e1a thru 0x750115d (0xf344 bytes) 2024-04-23T23:50:52.318 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:52.332+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:52.319 INFO:teuthology.orchestra.run.smithi092.stdout:5142 trunc from 0x750115e to 0x882ac77 2024-04-23T23:50:52.324 INFO:teuthology.orchestra.run.smithi092.stdout:5143 read 0x7d0f492 thru 0x7d17dc9 (0x8938 bytes) 2024-04-23T23:50:52.324 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.324 INFO:teuthology.orchestra.run.smithi092.stdout:5145 read 0x6c81a94 thru 0x6c8c773 (0xace0 bytes) 2024-04-23T23:50:52.325 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.325 INFO:teuthology.orchestra.run.smithi092.stdout:5147 read 0x7c1a74c thru 0x7c208c3 (0x6178 bytes) 2024-04-23T23:50:52.336 INFO:teuthology.orchestra.run.smithi092.stdout:5149 writesame 0xbaaaf09 thru 0xbaac826 data_size 0x5(0x191e bytes) 2024-04-23T23:50:52.336 INFO:teuthology.orchestra.run.smithi092.stdout:5150 read 0x64db4b5 thru 0x64e0953 (0x549f bytes) 2024-04-23T23:50:52.336 INFO:teuthology.orchestra.run.smithi092.stdout:5151 punch from 0x257a18c to 0x25832c4, (0x9138 bytes) 2024-04-23T23:50:52.336 INFO:teuthology.orchestra.run.smithi092.stdout:5152 read 0x91bdefc thru 0x91c1265 (0x336a bytes) 2024-04-23T23:50:52.345 INFO:teuthology.orchestra.run.smithi092.stdout:5153 trunc from 0xbaac827 to 0x4dd98fd 2024-04-23T23:50:52.368 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:52.368 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:52.368 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:50:52.368 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:52.371 INFO:teuthology.orchestra.run.smithi092.stdout:5154 writesame 0x9f6bea3 thru 0x9f6fb6a data_size 0x14(0x3cc8 bytes) 2024-04-23T23:50:52.372 INFO:teuthology.orchestra.run.smithi092.stdout:5155 read 0x10d932a thru 0x10e0593 (0x726a bytes) 2024-04-23T23:50:52.372 INFO:teuthology.orchestra.run.smithi092.stdout:5156 read 0x70fc0b7 thru 0x70ff7a0 (0x36ea bytes) 2024-04-23T23:50:52.372 INFO:teuthology.orchestra.run.smithi092.stdout:5158 write 0x11ec058 thru 0x11f4c8e (0x8c37 bytes) 2024-04-23T23:50:52.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:52.382 INFO:teuthology.orchestra.run.smithi092.stdout:5159 trunc from 0x9f6fb6b to 0xebd74f3 2024-04-23T23:50:52.388 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.389 INFO:teuthology.orchestra.run.smithi092.stdout:5161 write 0x474ebd1 thru 0x474f932 (0xd62 bytes) 2024-04-23T23:50:52.392 INFO:teuthology.orchestra.run.smithi092.stdout:5163 write 0x16fce9b thru 0x170111b (0x4281 bytes) 2024-04-23T23:50:52.396 INFO:teuthology.orchestra.run.smithi092.stdout:5164 read 0x54f76ff thru 0x54f77fc (0xfe bytes) 2024-04-23T23:50:52.397 INFO:teuthology.orchestra.run.smithi092.stdout:5165 punch from 0x70d4783 to 0x70e0442, (0xbcbf bytes) 2024-04-23T23:50:52.397 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.397 INFO:teuthology.orchestra.run.smithi092.stdout:5167 write 0xdedbf24 thru 0xdee2462 (0x653f bytes) 2024-04-23T23:50:52.402 INFO:teuthology.orchestra.run.smithi092.stdout:5168 writesame 0x3833d18 thru 0x383f2a9 data_size 0x6(0xb592 bytes) 2024-04-23T23:50:52.409 INFO:teuthology.orchestra.run.smithi092.stdout:5169 write 0x647ce10 thru 0x648c466 (0xf657 bytes) 2024-04-23T23:50:52.416 INFO:teuthology.orchestra.run.smithi092.stdout:5171 write 0x888f9f thru 0x88f12c (0x618e bytes) 2024-04-23T23:50:52.417 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.417 INFO:teuthology.orchestra.run.smithi092.stdout:5175 writesame 0xd7ef11d thru 0xd7f76d3 data_size 0x1(0x85b7 bytes) 2024-04-23T23:50:52.419 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:52.432+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:52.441 INFO:teuthology.orchestra.run.smithi092.stdout:5176 write 0x3cde53e thru 0x3ced7cc (0xf28f bytes) 2024-04-23T23:50:52.449 INFO:teuthology.orchestra.run.smithi092.stdout:5177 write 0x3a745cc thru 0x3a7c345 (0x7d7a bytes) 2024-04-23T23:50:52.452 INFO:teuthology.orchestra.run.smithi092.stdout:5178 writesame 0xcc117c3 thru 0xcc11e49 data_size 0x1(0x687 bytes) 2024-04-23T23:50:52.456 INFO:teuthology.orchestra.run.smithi092.stdout:5179 read 0x1ed31d4 thru 0x1ee3135 (0xff62 bytes) 2024-04-23T23:50:52.456 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.457 INFO:teuthology.orchestra.run.smithi092.stdout:5182 punch from 0x1c087df to 0x1c15f20, (0xd741 bytes) 2024-04-23T23:50:52.457 INFO:teuthology.orchestra.run.smithi092.stdout:5183 read 0xe3ea6de thru 0xe3f8c4e (0xe571 bytes) 2024-04-23T23:50:52.457 INFO:teuthology.orchestra.run.smithi092.stdout:5184 writesame 0xd2ad3e2 thru 0xd2b4f20 data_size 0x1(0x7b3f bytes) 2024-04-23T23:50:52.481 INFO:teuthology.orchestra.run.smithi092.stdout:5186 write 0xca232ef thru 0xca2c3e8 (0x90fa bytes) 2024-04-23T23:50:52.491 INFO:teuthology.orchestra.run.smithi092.stdout:5187 write 0xaa4cdc2 thru 0xaa5763b (0xa87a bytes) 2024-04-23T23:50:52.499 INFO:teuthology.orchestra.run.smithi092.stdout:5188 writesame 0x925201d thru 0x925f326 data_size 0x2(0xd30a bytes) 2024-04-23T23:50:52.519 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.520 INFO:teuthology.orchestra.run.smithi092.stdout:5190 read 0x59fa26e thru 0x59fbe52 (0x1be5 bytes) 2024-04-23T23:50:52.520 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:52.533+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:52.520 INFO:teuthology.orchestra.run.smithi092.stdout:5193 read 0x36a8aa1 thru 0x36b6e74 (0xe3d4 bytes) 2024-04-23T23:50:52.520 INFO:teuthology.orchestra.run.smithi092.stdout:5194 write 0xc838a71 thru 0xc8452ce (0xc85e bytes) 2024-04-23T23:50:52.523 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:52.523 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:50:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:50:52.524 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:50:52.524 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:52.526 INFO:teuthology.orchestra.run.smithi092.stdout:5195 read 0xa56590e thru 0xa570dc7 (0xb4ba bytes) 2024-04-23T23:50:52.526 INFO:teuthology.orchestra.run.smithi092.stdout:5197 punch from 0x238c65d to 0x2396c13, (0xa5b6 bytes) 2024-04-23T23:50:52.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:52.540 INFO:teuthology.orchestra.run.smithi092.stdout:5198 trunc from 0xebd74f3 to 0x2e3cc37 2024-04-23T23:50:52.572 INFO:teuthology.orchestra.run.smithi092.stdout:5199 write 0x47d72fc thru 0x47e52fa (0xdfff bytes) 2024-04-23T23:50:52.586 INFO:teuthology.orchestra.run.smithi092.stdout:5200 writesame 0x57b97f4 thru 0x57c79f3 data_size 0x10(0xe200 bytes) 2024-04-23T23:50:52.596 INFO:teuthology.orchestra.run.smithi092.stdout:5201 trunc from 0x57c79f4 to 0xdbe220a 2024-04-23T23:50:52.601 INFO:teuthology.orchestra.run.smithi092.stdout:5202 read 0x762ee9a thru 0x7633952 (0x4ab9 bytes) 2024-04-23T23:50:52.602 INFO:teuthology.orchestra.run.smithi092.stdout:5203 write 0x715363b thru 0x7156a87 (0x344d bytes) 2024-04-23T23:50:52.606 INFO:teuthology.orchestra.run.smithi092.stdout:5204 read 0xa92955b thru 0xa931b0f (0x85b5 bytes) 2024-04-23T23:50:52.606 INFO:teuthology.orchestra.run.smithi092.stdout:5205 punch from 0x8f7be68 to 0x8f8039f, (0x4537 bytes) 2024-04-23T23:50:52.615 INFO:teuthology.orchestra.run.smithi092.stdout:5206 trunc from 0xdbe220a to 0x62b22aa 2024-04-23T23:50:52.625 INFO:teuthology.orchestra.run.smithi092.stdout:5207 punch from 0x60a08a3 to 0x60a2ef9, (0x2656 bytes) 2024-04-23T23:50:52.625 INFO:teuthology.orchestra.run.smithi092.stdout:5209 write 0x6168429 thru 0x617149d (0x9075 bytes) 2024-04-23T23:50:52.630 INFO:teuthology.orchestra.run.smithi092.stdout:5212 punch from 0x4acc10b to 0x4acd37a, (0x126f bytes) 2024-04-23T23:50:52.630 INFO:teuthology.orchestra.run.smithi092.stdout:5213 read 0x3739b1e thru 0x373bdcb (0x22ae bytes) 2024-04-23T23:50:52.630 INFO:teuthology.orchestra.run.smithi092.stdout:5214 writesame 0x68741f thru 0x690d53 data_size 0xd(0x9935 bytes) 2024-04-23T23:50:52.633 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:52.633 INFO:teuthology.orchestra.run.smithi092.stdout:5216 read 0x68f2f6 thru 0x694728 (0x5433 bytes) 2024-04-23T23:50:52.634 INFO:teuthology.orchestra.run.smithi092.stdout:5217 write 0x15d1049 thru 0x15dca4c (0xba04 bytes) 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.457045784000002, 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.646 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:50:52.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:50:52.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.108309116000001, 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:52.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:50:52.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.108292663, 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:50:52.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.108265922000001, 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:50:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.108247126999998, 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.108229543, 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:50:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:50:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.096534038999998, 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:50:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.096358352000003, 2024-04-23T23:50:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.096206832999997, 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:50:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:50:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.095924207000003, 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.095778342000003, 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:50:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:50:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:50:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:50:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.094828657999997, 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.094805764, 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:50:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:50:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:50:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.078790388999998, 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:50:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.078780754999997, 2024-04-23T23:50:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:50:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:50:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.063748394000001, 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:50:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.063727069999999, 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:50:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.063704244999997, 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:50:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:50:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.063681566, 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:50:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.061379842999997, 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:50:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:50:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:50:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:50:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:50:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:50:52.737 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:52.635+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:52.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:52.739 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:52.751+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:52.739 INFO:teuthology.orchestra.run.smithi092.stdout:5218 write 0x975ea78 thru 0x976df03 (0xf48c bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5219 writesame 0x84a3981 thru 0x84b35f9 data_size 0x1(0xfc79 bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5220 trunc from 0x976df04 to 0x669bd8c 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5222 writesame 0x9c2cce0 thru 0x9c395ee data_size 0x3f(0xc90f bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5223 read 0x6aa495e thru 0x6aa98b7 (0x4f5a bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5224 read 0x9c3805f thru 0x9c395ee (0x1590 bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5225 write 0x2464f5f thru 0x246ab7e (0x5c20 bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5226 read 0x98cc190 thru 0x98cef31 (0x2da2 bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5228 read 0x49ecdfb thru 0x49f14d4 (0x46da bytes) 2024-04-23T23:50:52.740 INFO:teuthology.orchestra.run.smithi092.stdout:5229 clone 33 order 21 su 1048576 sc 4 2024-04-23T23:50:52.838 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:52.852+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:52.842 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:52.842 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:52.842 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:52.842 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:52.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:52.939 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:52.953+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:52.993 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:52.994 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:53.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:53.040 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:53.054+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:53.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.003430113, 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.000835781999999, 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:50:53.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.116 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.889180343, 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:50:53.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:50:53.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.889161903000002, 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:53.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:50:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.889142797000002, 2024-04-23T23:50:53.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:50:53.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.889124288000001, 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:50:53.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:50:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.889021239000002, 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:50:53.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:50:53.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.889001619999998, 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:50:53.138 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888896145, 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:53.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888870959999998, 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:50:53.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888837415000001, 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:53.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:50:53.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:50:53.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888813652, 2024-04-23T23:50:53.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:50:53.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888792611, 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:50:53.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:50:53.163 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888724375999999, 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888699908, 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:53.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888674210000001, 2024-04-23T23:50:53.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:50:53.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:50:53.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.176 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888652444000002, 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:50:53.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888631977999999, 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.181 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888554118999998, 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.185 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.888524468, 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:53.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:50:53.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:50:53.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.194 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:53.195 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:53.209+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:53.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:53.213 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T23:50:53.213 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:50:53.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-23T23:50:53.296 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:53.310+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:53.324 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:53.324 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:53.324 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:53.324 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:53.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:53.393 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-04-23T23:50:53.397 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:53.410+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:53.407 INFO:teuthology.orchestra.run.smithi092.stdout:5230 trunc from 0x9c395ef to 0xc8292e7 2024-04-23T23:50:53.421 INFO:teuthology.orchestra.run.smithi092.stdout:5231 trunc from 0xc8292e7 to 0xec40e57 2024-04-23T23:50:53.429 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:53.429 INFO:teuthology.orchestra.run.smithi092.stdout:5234 write 0xc91a8ec thru 0xc9275ea (0xccff bytes) 2024-04-23T23:50:53.440 INFO:teuthology.orchestra.run.smithi092.stdout:5235 trunc from 0xec40e57 to 0x89e587c 2024-04-23T23:50:53.458 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:53.458 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:53.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:53.471 INFO:teuthology.orchestra.run.smithi092.stdout:5236 writesame 0xbcc9b85 thru 0xbcd0a12 data_size 0x6a(0x6e8e bytes) 2024-04-23T23:50:53.471 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:53.471 INFO:teuthology.orchestra.run.smithi092.stdout:5238 write 0x154a439 thru 0x15509f0 (0x65b8 bytes) 2024-04-23T23:50:53.476 INFO:teuthology.orchestra.run.smithi092.stdout:5239 write 0x2306d23 thru 0x230718d (0x46b bytes) 2024-04-23T23:50:53.481 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:53.494 INFO:teuthology.orchestra.run.smithi092.stdout:5241 trunc from 0xbcd0a13 to 0x1475a9f 2024-04-23T23:50:53.498 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:53.512+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:53.518 INFO:teuthology.orchestra.run.smithi092.stdout:5242 read 0xfcdf5a thru 0xfdcc5d (0xed04 bytes) 2024-04-23T23:50:53.537 INFO:teuthology.orchestra.run.smithi092.stdout:5243 writesame 0xb9686e8 thru 0xb9730db data_size 0x49(0xa9f4 bytes) 2024-04-23T23:50:53.537 INFO:teuthology.orchestra.run.smithi092.stdout:5244 writesame 0x3ebde93 thru 0x3ebf9d6 data_size 0xa(0x1b44 bytes) 2024-04-23T23:50:53.540 INFO:teuthology.orchestra.run.smithi092.stdout:5245 read 0x62edd37 thru 0x62f3cff (0x5fc9 bytes) 2024-04-23T23:50:53.540 INFO:teuthology.orchestra.run.smithi092.stdout:5246 punch from 0x2d462fc to 0x2d53cc3, (0xd9c7 bytes) 2024-04-23T23:50:53.540 INFO:teuthology.orchestra.run.smithi092.stdout:5247 write 0x6a50d50 thru 0x6a59f87 (0x9238 bytes) 2024-04-23T23:50:53.543 INFO:teuthology.orchestra.run.smithi092.stdout:5248 write 0x2d7115 thru 0x2d75bb (0x4a7 bytes) 2024-04-23T23:50:53.558 INFO:teuthology.orchestra.run.smithi092.stdout:5250 trunc from 0xb9730dc to 0x9ca80e0 2024-04-23T23:50:53.564 INFO:teuthology.orchestra.run.smithi092.stdout:5251 read 0x7ffa905 thru 0x7ffc264 (0x1960 bytes) 2024-04-23T23:50:53.572 INFO:teuthology.orchestra.run.smithi092.stdout:5253 writesame 0xb21473e thru 0xb21b6d9 data_size 0x6(0x6f9c bytes) 2024-04-23T23:50:53.572 INFO:teuthology.orchestra.run.smithi092.stdout:5254 punch from 0x3182c4f to 0x31923ba, (0xf76b bytes) 2024-04-23T23:50:53.572 INFO:teuthology.orchestra.run.smithi092.stdout:5255 read 0x448bdfa thru 0x44958ba (0x9ac1 bytes) 2024-04-23T23:50:53.572 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:53.572 INFO:teuthology.orchestra.run.smithi092.stdout:5257 clone 34 order 23 su 1048576 sc 11 2024-04-23T23:50:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:53.591 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:53.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.788508419999999, 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.786464436999999, 2024-04-23T23:50:53.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:50:53.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:50:53.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.600 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.784487189, 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:53.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.604 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.784456555999999, 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.605 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:50:53.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.782513706, 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:50:53.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.780428496999999, 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:53.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.778419066000001, 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:50:53.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.776519098000001, 2024-04-23T23:50:53.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.772470569999999, 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:50:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:50:53.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.770425893999999, 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:50:53.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:50:53.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:50:53.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.768352320999998, 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:50:53.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:50:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.766464164999999, 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.766443387999999, 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:50:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:50:53.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:50:53.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:50:53.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:50:53.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.649 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.650 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:53.650 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.758451894, 2024-04-23T23:50:53.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:50:53.650 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:50:53.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:53.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:50:53.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:50:53.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:50:53.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:50:53.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:50:53.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:50:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:50:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:50:53.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:50:53.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:50:53.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:50:53.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.660 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.660 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:53.660 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.756516911999999, 2024-04-23T23:50:53.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:50:53.660 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:50:53.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:53.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:53.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:50:53.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:50:53.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:50:53.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:50:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:50:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:50:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:50:53.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:50:53.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:50:53.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:50:53.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:50:53.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:50:53.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:50:53.671 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.756490817, 2024-04-23T23:50:53.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.678 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.374773352999998, 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.374730933999999, 2024-04-23T23:50:53.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.374694977000001, 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:50:53.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:50:53.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.374670619, 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:50:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:50:53.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:50:53.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:50:53.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.697 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:53.697 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:53.697 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:53.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:53.700 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:53.712+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:53.799 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:53.813+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:53.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:53.843 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:53.843 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:53.843 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:53.843 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:53.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:53.900 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:53.914+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:53.931 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:53.949 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:53.949 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:53.966 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:53.966 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:53.983 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:53.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:53.999 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:53.999 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.001 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:54.001 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:54.001 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:54.015+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:54.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:54.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:54.036 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:54.055 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:54.055 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:54.102 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:54.116+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:54.125 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.125 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:54.203 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:54.217+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-04-23T23:50:54.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-23T23:50:54.240 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.240 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:54.282 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone33 from 0xb21b6da (overlap 0x1475a9f) to 0x1269e5b 2024-04-23T23:50:54.304 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:54.318+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-04-23T23:50:54.405 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.405 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.405 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:54.419+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:54.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:54.506 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:54.519+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:54.554 INFO:teuthology.orchestra.run.smithi092.stdout:5258 read 0x8211efa thru 0x821c39d (0xa4a4 bytes) 2024-04-23T23:50:54.556 INFO:teuthology.orchestra.run.smithi092.stdout:5261 trunc from 0xb21b6da to 0xcec080a 2024-04-23T23:50:54.565 INFO:teuthology.orchestra.run.smithi092.stdout:5263 read 0x1fb8a98 thru 0x1fc485d (0xbdc6 bytes) 2024-04-23T23:50:54.565 INFO:teuthology.orchestra.run.smithi092.stdout:5264 punch from 0x3b26542 to 0x3b29b4e, (0x360c bytes) 2024-04-23T23:50:54.565 INFO:teuthology.orchestra.run.smithi092.stdout:5266 write 0xc50702a thru 0xc50bdd2 (0x4da9 bytes) 2024-04-23T23:50:54.567 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.567 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.569 INFO:teuthology.orchestra.run.smithi092.stdout:5267 write 0x5cfab48 thru 0x5d03530 (0x89e9 bytes) 2024-04-23T23:50:54.573 INFO:teuthology.orchestra.run.smithi092.stdout:5269 punch from 0x125881f to 0x12591d6, (0x9b7 bytes) 2024-04-23T23:50:54.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:54.585 INFO:teuthology.orchestra.run.smithi092.stdout:5272 trunc from 0xcec080a to 0x2634999 2024-04-23T23:50:54.606 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:54.620+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:54.636 INFO:teuthology.orchestra.run.smithi092.stdout:5276 writesame 0x86ae7ce thru 0x86af0f3 data_size 0x1(0x926 bytes) 2024-04-23T23:50:54.649 INFO:teuthology.orchestra.run.smithi092.stdout:5278 writesame 0xecf0aaa thru 0xecfa15e data_size 0x1(0x96b5 bytes) 2024-04-23T23:50:54.649 INFO:teuthology.orchestra.run.smithi092.stdout:5280 punch from 0x7608881 to 0x7610d1b, (0x849a bytes) 2024-04-23T23:50:54.649 INFO:teuthology.orchestra.run.smithi092.stdout:5283 writesame 0x505b321 thru 0x506b205 data_size 0x1(0xfee5 bytes) 2024-04-23T23:50:54.690 INFO:teuthology.orchestra.run.smithi092.stdout:5285 read 0xe3d711c thru 0xe3dabf3 (0x3ad8 bytes) 2024-04-23T23:50:54.690 INFO:teuthology.orchestra.run.smithi092.stdout:5286 write 0x580b03d thru 0x580f3c2 (0x4386 bytes) 2024-04-23T23:50:54.693 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.693 INFO:teuthology.orchestra.run.smithi092.stdout:5288 read 0x2741b5 thru 0x27ecd6 (0xab22 bytes) 2024-04-23T23:50:54.693 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.694 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.694 INFO:teuthology.orchestra.run.smithi092.stdout:5290 writesame 0xdb3df9e thru 0xdb3e8af data_size 0x3(0x912 bytes) 2024-04-23T23:50:54.697 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.697 INFO:teuthology.orchestra.run.smithi092.stdout:5293 write 0x8786698 thru 0x8791930 (0xb299 bytes) 2024-04-23T23:50:54.699 INFO:teuthology.orchestra.run.smithi092.stdout:5295 write 0xd2f5dd8 thru 0xd2f5fa1 (0x1ca bytes) 2024-04-23T23:50:54.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:54.707 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:54.721+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:54.711 INFO:teuthology.orchestra.run.smithi092.stdout:5296 trunc from 0xecfa15f to 0x7e43444 2024-04-23T23:50:54.723 INFO:teuthology.orchestra.run.smithi092.stdout:5298 writesame 0x2281111 thru 0x22893c4 data_size 0x46(0x82b4 bytes) 2024-04-23T23:50:54.725 INFO:teuthology.orchestra.run.smithi092.stdout:5300 punch from 0x3f9309c to 0x3f96dee, (0x3d52 bytes) 2024-04-23T23:50:54.725 INFO:teuthology.orchestra.run.smithi092.stdout:5301 read 0x2e08f7a thru 0x2e1261e (0x96a5 bytes) 2024-04-23T23:50:54.726 INFO:teuthology.orchestra.run.smithi092.stdout:5302 punch from 0x1b166e5 to 0x1b22fa4, (0xc8bf bytes) 2024-04-23T23:50:54.732 INFO:teuthology.orchestra.run.smithi092.stdout:5303 write 0x971ce48 thru 0x97238e9 (0x6aa2 bytes) 2024-04-23T23:50:54.735 INFO:teuthology.orchestra.run.smithi092.stdout:5305 writesame 0x5abd752 thru 0x5ac6fdd data_size 0x1a(0x988c bytes) 2024-04-23T23:50:54.740 INFO:teuthology.orchestra.run.smithi092.stdout:5306 writesame 0x1b7e180 thru 0x1b81984 data_size 0x1(0x3805 bytes) 2024-04-23T23:50:54.753 INFO:teuthology.orchestra.run.smithi092.stdout:5307 trunc from 0x97238ea to 0x63cdba0 2024-04-23T23:50:54.761 INFO:teuthology.orchestra.run.smithi092.stdout:5308 punch from 0x2dab6ea to 0x2dbb420, (0xfd36 bytes) 2024-04-23T23:50:54.761 INFO:teuthology.orchestra.run.smithi092.stdout:5309 read 0x4f410b4 thru 0x4f42b6e (0x1abb bytes) 2024-04-23T23:50:54.761 INFO:teuthology.orchestra.run.smithi092.stdout:5311 read 0xf8468a thru 0xf917ca (0xd141 bytes) 2024-04-23T23:50:54.761 INFO:teuthology.orchestra.run.smithi092.stdout:5312 writesame 0x46e1ee9 thru 0x46f0f3b data_size 0x77(0xf053 bytes) 2024-04-23T23:50:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:5314 write 0xf7bc2 thru 0xff9e9 (0x7e28 bytes) 2024-04-23T23:50:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:5316 read 0x2b2dd69 thru 0x2b2ea5d (0xcf5 bytes) 2024-04-23T23:50:54.772 INFO:teuthology.orchestra.run.smithi092.stdout:5317 writesame 0x76c2660 thru 0x76cb1aa data_size 0xd(0x8b4b bytes) 2024-04-23T23:50:54.781 INFO:teuthology.orchestra.run.smithi092.stdout:5318 trunc from 0x76cb1ab to 0xdea98e6 2024-04-23T23:50:54.787 INFO:teuthology.orchestra.run.smithi092.stdout:5322 writesame 0xad8651 thru 0xadcc07 data_size 0x9(0x45b7 bytes) 2024-04-23T23:50:54.789 INFO:teuthology.orchestra.run.smithi092.stdout:5325 write 0x32d619c thru 0x32d852f (0x2394 bytes) 2024-04-23T23:50:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:5326 read 0x3d24c50 thru 0x3d31dd0 (0xd181 bytes) 2024-04-23T23:50:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:5327 read 0x990348 thru 0x993f2e (0x3be7 bytes) 2024-04-23T23:50:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:5328 writesame 0x9e3fb77 thru 0x9e4d059 data_size 0x1(0xd4e3 bytes) 2024-04-23T23:50:54.808 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:54.822+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:54.810 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.810 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:54.824 INFO:teuthology.orchestra.run.smithi092.stdout:5329 trunc from 0xdea98e6 to 0xb00db1b 2024-04-23T23:50:54.834 INFO:teuthology.orchestra.run.smithi092.stdout:5330 punch from 0xa430148 to 0xa432e97, (0x2d4f bytes) 2024-04-23T23:50:54.834 INFO:teuthology.orchestra.run.smithi092.stdout:5332 punch from 0x772c204 to 0x7737126, (0xaf22 bytes) 2024-04-23T23:50:54.837 INFO:teuthology.orchestra.run.smithi092.stdout:5333 trunc from 0xb00db1b to 0xe9f1fed 2024-04-23T23:50:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:5334 read 0xe7e43e2 thru 0xe7f2e6e (0xea8d bytes) 2024-04-23T23:50:54.851 INFO:teuthology.orchestra.run.smithi092.stdout:5335 trunc from 0xe9f1fed to 0x8404262 2024-04-23T23:50:54.858 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.858 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.858 INFO:teuthology.orchestra.run.smithi092.stdout:5340 punch from 0x3696927 to 0x369f298, (0x8971 bytes) 2024-04-23T23:50:54.866 INFO:teuthology.orchestra.run.smithi092.stdout:5342 trunc from 0x8404262 to 0x1e6974a 2024-04-23T23:50:54.879 INFO:teuthology.orchestra.run.smithi092.stdout:5344 punch from 0x7d3ee to 0x80e8c, (0x3a9e bytes) 2024-04-23T23:50:54.881 INFO:teuthology.orchestra.run.smithi092.stdout:5345 trunc from 0x1e6974a to 0x3869e48 2024-04-23T23:50:54.886 INFO:teuthology.orchestra.run.smithi092.stdout:5346 read 0x251586d thru 0x252005e (0xa7f2 bytes) 2024-04-23T23:50:54.886 INFO:teuthology.orchestra.run.smithi092.stdout:5347 punch from 0x39c53b to 0x39f395, (0x2e5a bytes) 2024-04-23T23:50:54.886 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.887 INFO:teuthology.orchestra.run.smithi092.stdout:5350 read 0x32c47fc thru 0x32cd35f (0x8b64 bytes) 2024-04-23T23:50:54.903 INFO:teuthology.orchestra.run.smithi092.stdout:5352 write 0xcb7d389 thru 0xcb7e4d7 (0x114f bytes) 2024-04-23T23:50:54.909 INFO:teuthology.orchestra.run.smithi092.stdout:5353 trunc from 0xcb7e4d8 to 0xa279561 2024-04-23T23:50:54.909 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:54.923+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:54.920 INFO:teuthology.orchestra.run.smithi092.stdout:5354 writesame 0x66284b0 thru 0x6631922 data_size 0x3d(0x9473 bytes) 2024-04-23T23:50:54.926 INFO:teuthology.orchestra.run.smithi092.stdout:5355 trunc from 0xa279561 to 0x7b6cd07 2024-04-23T23:50:54.936 INFO:teuthology.orchestra.run.smithi092.stdout:5357 trunc from 0x7b6cd07 to 0xa694d84 2024-04-23T23:50:54.941 INFO:teuthology.orchestra.run.smithi092.stdout:5358 write 0x2e94ebc thru 0x2ea3ea8 (0xefed bytes) 2024-04-23T23:50:54.942 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.942 INFO:teuthology.orchestra.run.smithi092.stdout:5361 read 0x4dae043 thru 0x4db3161 (0x511f bytes) 2024-04-23T23:50:54.943 INFO:teuthology.orchestra.run.smithi092.stdout:5362 writesame 0x1aa30d8 thru 0x1aad1d9 data_size 0x25(0xa102 bytes) 2024-04-23T23:50:54.945 INFO:teuthology.orchestra.run.smithi092.stdout:5363 write 0x9837377 thru 0x9843fc9 (0xcc53 bytes) 2024-04-23T23:50:54.948 INFO:teuthology.orchestra.run.smithi092.stdout:5364 read 0x52acb84 thru 0x52bb2db (0xe758 bytes) 2024-04-23T23:50:54.949 INFO:teuthology.orchestra.run.smithi092.stdout:5365 write 0x18d7dec thru 0x18e22ee (0xa503 bytes) 2024-04-23T23:50:54.950 INFO:teuthology.orchestra.run.smithi092.stdout:5366 write 0x6d14040 thru 0x6d1d258 (0x9219 bytes) 2024-04-23T23:50:54.953 INFO:teuthology.orchestra.run.smithi092.stdout:5367 read 0x18bbc9a thru 0x18c436e (0x86d5 bytes) 2024-04-23T23:50:54.954 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.962 INFO:teuthology.orchestra.run.smithi092.stdout:5369 writesame 0xc7a8cc5 thru 0xc7aac6d data_size 0x1(0x1fa9 bytes) 2024-04-23T23:50:54.962 INFO:teuthology.orchestra.run.smithi092.stdout:5370 write 0xa3949b6 thru 0xa3a427c (0xf8c7 bytes) 2024-04-23T23:50:54.964 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.964 INFO:teuthology.orchestra.run.smithi092.stdout:5373 write 0x5677ba6 thru 0x567c9b5 (0x4e10 bytes) 2024-04-23T23:50:54.965 INFO:teuthology.orchestra.run.smithi092.stdout:5374 punch from 0x5af98ff to 0x5afe22b, (0x492c bytes) 2024-04-23T23:50:54.965 INFO:teuthology.orchestra.run.smithi092.stdout:5375 write 0x712df47 thru 0x71340f4 (0x61ae bytes) 2024-04-23T23:50:54.969 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:54.970 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:54.974 INFO:teuthology.orchestra.run.smithi092.stdout:5376 trunc from 0xc7aac6e to 0x6695165 2024-04-23T23:50:54.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:54.988 INFO:teuthology.orchestra.run.smithi092.stdout:5377 write 0x8440765 thru 0x8442167 (0x1a03 bytes) 2024-04-23T23:50:54.991 INFO:teuthology.orchestra.run.smithi092.stdout:5380 trunc from 0x8442168 to 0x8206d25 2024-04-23T23:50:54.997 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.997 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:54.997 INFO:teuthology.orchestra.run.smithi092.stdout:5385 punch from 0x7ffa1d3 to 0x7fffd7f, (0x5bac bytes) 2024-04-23T23:50:55.009 INFO:teuthology.orchestra.run.smithi092.stdout:5388 write 0xdf054ef thru 0xdf0be2a (0x693c bytes) 2024-04-23T23:50:55.012 INFO:teuthology.orchestra.run.smithi092.stdout:5389 writesame 0x359d2ae thru 0x35a2ea7 data_size 0x2(0x5bfa bytes) 2024-04-23T23:50:55.020 INFO:teuthology.orchestra.run.smithi092.stdout:5390 write 0x38d35b6 thru 0x38de6aa (0xb0f5 bytes) 2024-04-23T23:50:55.022 INFO:teuthology.orchestra.run.smithi092.stdout:5392 writesame 0x4a40a4 thru 0x4a4687 data_size 0x1a(0x5e4 bytes) 2024-04-23T23:50:55.023 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:55.037+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:55.023 INFO:teuthology.orchestra.run.smithi092.stdout:5393 writesame 0x17376bf thru 0x174055b data_size 0x1(0x8e9d bytes) 2024-04-23T23:50:55.042 INFO:teuthology.orchestra.run.smithi092.stdout:5394 write 0x7bc845 thru 0x7cb0e7 (0xe8a3 bytes) 2024-04-23T23:50:55.047 INFO:teuthology.orchestra.run.smithi092.stdout:5396 trunc from 0xdf0be2b to 0xbac956b 2024-04-23T23:50:55.055 INFO:teuthology.orchestra.run.smithi092.stdout:5397 read 0x11cbc79 thru 0x11d613c (0xa4c4 bytes) 2024-04-23T23:50:55.055 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:55.055 INFO:teuthology.orchestra.run.smithi092.stdout:5399 read 0x23295c8 thru 0x232a19e (0xbd7 bytes) 2024-04-23T23:50:55.056 INFO:teuthology.orchestra.run.smithi092.stdout:5400 read 0x500b27d thru 0x5014ba7 (0x992b bytes) 2024-04-23T23:50:55.066 INFO:teuthology.orchestra.run.smithi092.stdout:5401 writesame 0xece2751 thru 0xece5f92 data_size 0x13(0x3842 bytes) 2024-04-23T23:50:55.069 INFO:teuthology.orchestra.run.smithi092.stdout:5402 trunc from 0xece5f93 to 0xbda07ce 2024-04-23T23:50:55.076 INFO:teuthology.orchestra.run.smithi092.stdout:5403 punch from 0xad4e3a3 to 0xad5a5e7, (0xc244 bytes) 2024-04-23T23:50:55.076 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:55.084 INFO:teuthology.orchestra.run.smithi092.stdout:5406 write 0xe7f3c52 thru 0xe7fcc89 (0x9038 bytes) 2024-04-23T23:50:55.086 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.086 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.087 INFO:teuthology.orchestra.run.smithi092.stdout:5407 punch from 0x75b7eec to 0x75c4a34, (0xcb48 bytes) 2024-04-23T23:50:55.087 INFO:teuthology.orchestra.run.smithi092.stdout:5408 writesame 0x2d0937c thru 0x2d0f72d data_size 0x7(0x63b2 bytes) 2024-04-23T23:50:55.091 INFO:teuthology.orchestra.run.smithi092.stdout:5409 read 0x1304458 thru 0x130d344 (0x8eed bytes) 2024-04-23T23:50:55.091 INFO:teuthology.orchestra.run.smithi092.stdout:5410 writesame 0x3b6e131 thru 0x3b739b0 data_size 0x40(0x5880 bytes) 2024-04-23T23:50:55.093 INFO:teuthology.orchestra.run.smithi092.stdout:5411 clone 35 order 18 su 8192 sc 15 2024-04-23T23:50:55.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:55.136 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:55.150+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:55.213 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:55.226+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:55.217 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-04-23T23:50:55.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:50:55.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-23T23:50:55.240 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.240 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:55.277 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone34 intact 2024-04-23T23:50:55.318 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:55.332+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:55.353 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.353 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:55.415 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:55.429+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:55.462 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-04-23T23:50:55.468 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.469 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:55.506 INFO:teuthology.orchestra.run.smithi092.stdout:5412 write 0x660e43d thru 0x661a5eb (0xc1af bytes) 2024-04-23T23:50:55.515 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:55.529+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:55.531 INFO:teuthology.orchestra.run.smithi092.stdout:5413 write 0x11a80db thru 0x11b5413 (0xd339 bytes) 2024-04-23T23:50:55.553 INFO:teuthology.orchestra.run.smithi092.stdout:5414 trunc from 0xe7fcc8a to 0xbb37334 2024-04-23T23:50:55.584 INFO:teuthology.orchestra.run.smithi092.stdout:5415 read 0x73c8699 thru 0x73d0ca4 (0x860c bytes) 2024-04-23T23:50:55.584 INFO:teuthology.orchestra.run.smithi092.stdout:5416 punch from 0xb0f1443 to 0xb0fb3e7, (0x9fa4 bytes) 2024-04-23T23:50:55.584 INFO:teuthology.orchestra.run.smithi092.stdout:5417 trunc from 0xbb37334 to 0xb5c3581 2024-04-23T23:50:55.617 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:55.631+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:55.621 INFO:teuthology.orchestra.run.smithi092.stdout:5418 trunc from 0xb5c3581 to 0x5a7ecae 2024-04-23T23:50:55.636 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.636 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.645 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:55.668 INFO:teuthology.orchestra.run.smithi092.stdout:5420 read 0xf0ceaf thru 0xf0d41b (0x56d bytes) 2024-04-23T23:50:55.678 INFO:teuthology.orchestra.run.smithi092.stdout:5421 writesame 0x895dfca thru 0x896cf92 data_size 0x1(0xefc9 bytes) 2024-04-23T23:50:55.678 INFO:teuthology.orchestra.run.smithi092.stdout:5422 punch from 0xfb7d00 to 0xfb82bf, (0x5bf bytes) 2024-04-23T23:50:55.685 INFO:teuthology.orchestra.run.smithi092.stdout:5423 trunc from 0x896cf93 to 0x2c7198e 2024-04-23T23:50:55.717 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:55.731+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:55.735 INFO:teuthology.orchestra.run.smithi092.stdout:5424 trunc from 0x2c7198e to 0xc6d214b 2024-04-23T23:50:55.741 INFO:teuthology.orchestra.run.smithi092.stdout:5426 punch from 0x4512473 to 0x451d99f, (0xb52c bytes) 2024-04-23T23:50:55.741 INFO:teuthology.orchestra.run.smithi092.stdout:5428 read 0x1a0108b thru 0x1a045d3 (0x3549 bytes) 2024-04-23T23:50:55.741 INFO:teuthology.orchestra.run.smithi092.stdout:5430 read 0xb851b0f thru 0xb8600c5 (0xe5b7 bytes) 2024-04-23T23:50:55.741 INFO:teuthology.orchestra.run.smithi092.stdout:5431 read 0x15b089f thru 0x15b1ccd (0x142f bytes) 2024-04-23T23:50:55.742 INFO:teuthology.orchestra.run.smithi092.stdout:5432 read 0x80ba655 thru 0x80bcedf (0x288b bytes) 2024-04-23T23:50:55.742 INFO:teuthology.orchestra.run.smithi092.stdout:5433 write 0x9535107 thru 0x954476b (0xf665 bytes) 2024-04-23T23:50:55.759 INFO:teuthology.orchestra.run.smithi092.stdout:5434 write 0xde6e3c8 thru 0xde761a6 (0x7ddf bytes) 2024-04-23T23:50:55.767 INFO:teuthology.orchestra.run.smithi092.stdout:5438 punch from 0x1810182 to 0x181449f, (0x431d bytes) 2024-04-23T23:50:55.767 INFO:teuthology.orchestra.run.smithi092.stdout:5440 read 0x12a835a thru 0x12b50fc (0xcda3 bytes) 2024-04-23T23:50:55.769 INFO:teuthology.orchestra.run.smithi092.stdout:5441 write 0xa76b8ff thru 0xa76be58 (0x55a bytes) 2024-04-23T23:50:55.777 INFO:teuthology.orchestra.run.smithi092.stdout:5442 write 0xdedadea thru 0xdee8c69 (0xde80 bytes) 2024-04-23T23:50:55.797 INFO:teuthology.orchestra.run.smithi092.stdout:5446 trunc from 0xdee8c6a to 0x285e9b0 2024-04-23T23:50:55.802 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.802 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:50:55.838 INFO:teuthology.orchestra.run.smithi092.stdout:5447 read 0x22bba7 thru 0x23617b (0xa5d5 bytes) 2024-04-23T23:50:55.856 INFO:teuthology.orchestra.run.smithi092.stdout:5448 write 0xbba2b32 thru 0xbba2bd5 (0xa4 bytes) 2024-04-23T23:50:55.858 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:55.872+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:55.859 INFO:teuthology.orchestra.run.smithi092.stdout:5449 writesame 0x9a41207 thru 0x9a4de6a data_size 0x4(0xcc64 bytes) 2024-04-23T23:50:55.870 INFO:teuthology.orchestra.run.smithi092.stdout:5450 read 0x921bc20 thru 0x921d142 (0x1523 bytes) 2024-04-23T23:50:55.870 INFO:teuthology.orchestra.run.smithi092.stdout:5451 write 0x7ae7de3 thru 0x7aee4f2 (0x6710 bytes) 2024-04-23T23:50:55.882 INFO:teuthology.orchestra.run.smithi092.stdout:5453 trunc from 0xbba2bd6 to 0x6468154 2024-04-23T23:50:55.895 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:55.896 INFO:teuthology.orchestra.run.smithi092.stdout:5456 punch from 0x10311ae to 0x103acdd, (0x9b2f bytes) 2024-04-23T23:50:55.896 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:5458 write 0x6d678f0 thru 0x6d7307c (0xb78d bytes) 2024-04-23T23:50:55.910 INFO:teuthology.orchestra.run.smithi092.stdout:5460 writesame 0x1f376fb thru 0x1f37872 data_size 0x4(0x178 bytes) 2024-04-23T23:50:55.918 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:55.932+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:55.922 INFO:teuthology.orchestra.run.smithi092.stdout:5461 write 0x89cf10a thru 0x89dd203 (0xe0fa bytes) 2024-04-23T23:50:55.928 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:55.928 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:55.931 INFO:teuthology.orchestra.run.smithi092.stdout:5467 punch from 0x4530e11 to 0x4540c0f, (0xfdfe bytes) 2024-04-23T23:50:55.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:55.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:50:55.938 INFO:teuthology.orchestra.run.smithi092.stdout:5470 writesame 0x8c9c86f thru 0x8ca99ad data_size 0x17(0xd13f bytes) 2024-04-23T23:50:55.938 INFO:teuthology.orchestra.run.smithi092.stdout:5471 read 0x9ccce3 thru 0x9d3344 (0x6662 bytes) 2024-04-23T23:50:55.938 INFO:teuthology.orchestra.run.smithi092.stdout:5472 clone 36 order 23 su 2097152 sc 13 2024-04-23T23:50:56.019 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:56.033+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:56.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:50:56.079 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:56.079 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:56.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:50:56.134 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:56.145+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:56.186 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:56.187 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:56.192 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:56.192 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:56.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:50:56.203 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:56.204 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:56.222 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-04-23T23:50:56.222 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:50:56.223 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:56.236+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:56.223 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:56.223 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:56.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-23T23:50:56.250 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:56.250 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:56.268 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:56.269 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:56.287 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:56.288 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:56.302 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone35 from 0x8ca99ae (overlap 0x285e9b0) to 0x204baf3 2024-04-23T23:50:56.306 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:56.307 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:56.322 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:56.336+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:56.336 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:56.336 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101931 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344758+0000", 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0035438879999999998, 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035656559999999999, 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101931 2024-04-23T23:50:56.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344758+0000", 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344758+0000", 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344758+0000", 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3799999999999999e-07 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344759+0000", 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0419999999999999e-06 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344770+0000", 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0538e-05 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344773+0000", 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.083e-06 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101932 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344777+0000", 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0035250199999999998, 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036163950000000001, 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101932 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344777+0000", 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344777+0000", 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344777+0000", 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.72e-07 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344780+0000", 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7590000000000002e-06 2024-04-23T23:50:56.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344785+0000", 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6409999999999999e-06 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344790+0000", 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5519999999999998e-06 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101933 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344798+0000", 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0035043079999999998, 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036553409999999999, 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101933 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344798+0000", 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344798+0000", 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344798+0000", 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.82e-07 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344800+0000", 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1169999999999998e-06 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344804+0000", 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3499999999999999e-06 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344808+0000", 2024-04-23T23:50:56.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1750000000000001e-06 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101935 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344811+0000", 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0034905079999999998, 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036971119999999998, 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101935 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344811+0000", 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344811+0000", 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344812+0000", 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4900000000000002e-07 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344813+0000", 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.221e-06 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344816+0000", 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.089e-06 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344819+0000", 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101936 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344821+0000", 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0034805309999999998, 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037451630000000001, 2024-04-23T23:50:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101936 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344821+0000", 2024-04-23T23:50:56.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344821+0000", 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344822+0000", 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2600000000000001e-07 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344825+0000", 2024-04-23T23:50:56.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4769999999999999e-06 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344828+0000", 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2509999999999999e-06 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344831+0000", 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.757e-06 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.353 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101957 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344834+0000", 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0034673899999999999, 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037924009999999999, 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101957 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344834+0000", 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344834+0000", 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344835+0000", 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2024-04-23T23:50:56.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344836+0000", 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0839999999999999e-06 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344839+0000", 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0479999999999999e-06 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344842+0000", 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8009999999999999e-06 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101958 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344844+0000", 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0034574860000000001, 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038385649999999999, 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101958 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344844+0000", 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344844+0000", 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344845+0000", 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344846+0000", 2024-04-23T23:50:56.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344849+0000", 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344853+0000", 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9119999999999997e-06 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101959 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344873+0000", 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0034285769999999999, 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038657629999999999, 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101959 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344873+0000", 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344873+0000", 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344874+0000", 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2500000000000001e-07 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344875+0000", 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344882+0000", 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7410000000000003e-06 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344887+0000", 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0690000000000001e-06 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101972 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344895+0000", 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003407172, 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039021139999999999, 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101972 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344895+0000", 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344895+0000", 2024-04-23T23:50:56.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344895+0000", 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344897+0000", 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344904+0000", 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2489999999999998e-06 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344908+0000", 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5870000000000003e-06 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101973 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.367 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344915+0000", 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003387284, 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039397369999999996, 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101973 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344915+0000", 2024-04-23T23:50:56.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344915+0000", 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344915+0000", 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344917+0000", 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3140000000000002e-06 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344924+0000", 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3459999999999999e-06 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344928+0000", 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8219999999999997e-06 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101974 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344934+0000", 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003368079, 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039769430000000001, 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101974 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344934+0000", 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344934+0000", 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344934+0000", 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4900000000000001e-07 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344936+0000", 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1849999999999998e-06 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344942+0000", 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7450000000000001e-06 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344947+0000", 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9259999999999999e-06 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101975 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344953+0000", 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0033487920000000002, 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040226919999999996, 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101975 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344953+0000", 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344953+0000", 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344953+0000", 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5199999999999998e-07 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344956+0000", 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1289999999999999e-06 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344969+0000", 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3512e-05 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344974+0000", 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101976 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344990+0000", 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0033119389999999999, 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040435360000000004, 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101976 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344990+0000", 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344990+0000", 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344990+0000", 2024-04-23T23:50:56.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344993+0000", 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3750000000000001e-06 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345000+0000", 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8530000000000003e-06 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345005+0000", 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2429999999999996e-06 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101977 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345012+0000", 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0032902119999999998, 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040805809999999998, 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101977 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345012+0000", 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345012+0000", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345012+0000", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345014+0000", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.351e-06 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345021+0000", 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345031+0000", 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0560999999999999e-05 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101978 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345038+0000", 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0032639539999999999, 2024-04-23T23:50:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041122090000000004, 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101978 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345038+0000", 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345038+0000", 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345038+0000", 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.08e-07 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345041+0000", 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2299999999999998e-06 2024-04-23T23:50:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345047+0000", 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2179999999999996e-06 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345056+0000", 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8540000000000003e-06 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101979 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345062+0000", 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0032398930000000002, 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004145358, 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101979 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345062+0000", 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345062+0000", 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345062+0000", 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.82e-07 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345065+0000", 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1280000000000002e-06 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345070+0000", 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7400000000000001e-06 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345075+0000", 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8210000000000001e-06 2024-04-23T23:50:56.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102004 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345130+0000", 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0031718559999999998, 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004134146, 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102004 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345130+0000", 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345130+0000", 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345130+0000", 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0100000000000005e-07 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345133+0000", 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3159999999999999e-06 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345140+0000", 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0559999999999999e-06 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345145+0000", 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102005 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345151+0000", 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0031503799999999999, 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00417367, 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102005 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345151+0000", 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345151+0000", 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345152+0000", 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345154+0000", 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0379999999999998e-06 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345161+0000", 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345166+0000", 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102006 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345172+0000", 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003130121, 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042092199999999996, 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102006 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345172+0000", 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345172+0000", 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345172+0000", 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:56.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345174+0000", 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345186+0000", 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2143999999999999e-05 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345192+0000", 2024-04-23T23:50:56.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102007 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345198+0000", 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0031037209999999998, 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042388, 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102007 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345198+0000", 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345198+0000", 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345199+0000", 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345201+0000", 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1040000000000001e-06 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345207+0000", 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8880000000000002e-06 2024-04-23T23:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345211+0000", 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102008 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345218+0000", 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0030837410000000001, 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042752409999999999, 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102008 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345218+0000", 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345218+0000", 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345218+0000", 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345221+0000", 2024-04-23T23:50:56.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1919999999999999e-06 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345226+0000", 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6239999999999999e-06 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345231+0000", 2024-04-23T23:50:56.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102009 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345252+0000", 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0030496709999999999, 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043097459999999997, 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102009 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345252+0000", 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345252+0000", 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345253+0000", 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345255+0000", 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3089999999999998e-06 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345262+0000", 2024-04-23T23:50:56.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5220000000000004e-06 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345267+0000", 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3229999999999997e-06 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102010 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345273+0000", 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0030284940000000001, 2024-04-23T23:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043436539999999997, 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102010 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345273+0000", 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345273+0000", 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345274+0000", 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9299999999999998e-07 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345276+0000", 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2689999999999998e-06 2024-04-23T23:50:56.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345282+0000", 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345288+0000", 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3770000000000004e-06 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102011 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.345294+0000", 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0030082109999999998, 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043799370000000004, 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102011 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345294+0000", 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345294+0000", 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345294+0000", 2024-04-23T23:50:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345296+0000", 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0839999999999999e-06 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345302+0000", 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-06 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.345307+0000", 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6249999999999998e-06 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101925 3.b 3:d05d6d04:::rbd_data.11794a1e34ce.0000000000000001:25 [read 3842048~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344727+0000", 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0035749279999999998, 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050124119999999999, 2024-04-23T23:50:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101925 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344727+0000", 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344727+0000", 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344727+0000", 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8299999999999998e-07 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344729+0000", 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5209999999999999e-06 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344732+0000", 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7400000000000002e-06 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344735+0000", 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7719999999999999e-06 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349629+0000", 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004893695 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101926 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344738+0000", 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0035633349999999999, 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050774210000000004, 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101926 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344738+0000", 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344738+0000", 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344739+0000", 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3699999999999999e-07 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344740+0000", 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.085e-06 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344743+0000", 2024-04-23T23:50:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0649999999999999e-06 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344746+0000", 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6620000000000001e-06 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:102056 3.1e 3:7a1fa351:::rbd_data.1179644cb551.000000000000009e:head [stat out=16b,delete] snapc 26=[26] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.348671+0000", 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.9996309, 2024-04-23T23:50:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001209019, 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 102056 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348671+0000", 2024-04-23T23:50:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348671+0000", 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348672+0000", 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2389999999999999e-06 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348677+0000", 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0019999999999997e-06 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348687+0000", 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0278999999999999e-05 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348694+0000", 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.951e-06 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348755+0000", 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0427000000000003e-05 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.348832+0000", 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7612000000000002e-05 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349028+0000", 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019525900000000001 2024-04-23T23:50:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:101927 3.b 3.20b6ba0b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.344748+0000", 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003553358, 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005204714, 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 101927 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344748+0000", 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344748+0000", 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344749+0000", 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4200000000000002e-07 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344750+0000", 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.879999999999999e-07 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344753+0000", 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9009999999999998e-06 2024-04-23T23:50:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.344755+0000", 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5919999999999999e-06 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:102055 3.15 e77/56 3:a813d166:::rbd_data.1179644cb551.000000000000009d:head v 77'262, mlcod=76'259)", 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.349208+0000", 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.999094, 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081003200000000003, 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349208+0000", 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349208+0000", 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349208+0000", 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2500000000000005e-07 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349215+0000", 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349229+0000", 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3929e-05 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349236+0000", 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8199999999999997e-06 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349328+0000", 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1199000000000006e-05 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.349346+0000", 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8771e-05 2024-04-23T23:50:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 29 2024-04-23T23:50:56.437 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:56.438 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:56.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:50:56.440 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:56.452+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:56.466 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-04-23T23:50:56.539 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:56.553+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:56.590 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:56.590 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:56.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:50:56.643 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:56.657+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.605889591, 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:50:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:50:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.603295259999999, 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:50:56.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:50:56.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491639821, 2024-04-23T23:50:56.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:50:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:50:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491621381000002, 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:50:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:50:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491602274999998, 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:50:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:50:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491583766000002, 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:50:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:50:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:50:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491480717000002, 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:50:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491461097999998, 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:50:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:50:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:50:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491355623, 2024-04-23T23:50:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491330437999999, 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:50:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491296893000001, 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:50:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:50:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.49127313, 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:50:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:50:56.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491252089, 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:50:56.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491183853999999, 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:50:56.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491159386, 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491133688000001, 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:50:56.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:50:56.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491111922000002, 2024-04-23T23:50:56.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:50:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:56.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:50:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:50:56.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:50:56.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:50:56.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:50:56.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:50:56.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:50:56.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:56.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491091455999999, 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:50:56.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.800 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.491013596999998, 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:56.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:50:56.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:50:56.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:50:56.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:50:56.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:50:56.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:56.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:56.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.490983946, 2024-04-23T23:50:56.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:50:56.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.814 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:56.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:50:56.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:56.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:50:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.820 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:56.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5475 read 0xe7f72b thru 0xe8aed3 (0xb7a9 bytes) 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5476 trunc from 0x8ca99ae to 0xa81f15b 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5477 write 0xd6c2f06 thru 0xd6cb12b (0x8226 bytes) 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5478 read 0x7d0b2c2 thru 0x7d0e988 (0x36c7 bytes) 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5479 read 0xa60c231 thru 0xa615664 (0x9434 bytes) 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5483 trunc from 0xd6cb12c to 0xbd05c28 2024-04-23T23:50:56.823 INFO:teuthology.orchestra.run.smithi092.stdout:5484 write 0x5c63650 thru 0x5c6f0d0 (0xba81 bytes) 2024-04-23T23:50:56.824 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:56.835+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:56.834 INFO:teuthology.orchestra.run.smithi092.stdout:5487 read 0x1e2d9c8 thru 0x1e2dcfd (0x336 bytes) 2024-04-23T23:50:56.835 INFO:teuthology.orchestra.run.smithi092.stdout:5488 read 0x3336432 thru 0x33370eb (0xcba bytes) 2024-04-23T23:50:56.835 INFO:teuthology.orchestra.run.smithi092.stdout:5489 write 0xa2a57f3 thru 0xa2b016e (0xa97c bytes) 2024-04-23T23:50:56.836 INFO:teuthology.orchestra.run.smithi092.stdout:5491 read 0xae984d8 thru 0xaea15a8 (0x90d1 bytes) 2024-04-23T23:50:56.836 INFO:teuthology.orchestra.run.smithi092.stdout:5492 write 0x2c26c3e thru 0x2c2e74f (0x7b12 bytes) 2024-04-23T23:50:56.857 INFO:teuthology.orchestra.run.smithi092.stdout:5493 writesame 0x62bfbde thru 0x62c030e data_size 0x1(0x731 bytes) 2024-04-23T23:50:56.885 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:56.885 INFO:teuthology.orchestra.run.smithi092.stdout:5495 read 0x577b68f thru 0x5782d8d (0x76ff bytes) 2024-04-23T23:50:56.885 INFO:teuthology.orchestra.run.smithi092.stdout:5497 write 0x9099e53 thru 0x90a6253 (0xc401 bytes) 2024-04-23T23:50:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:56.898 INFO:teuthology.orchestra.run.smithi092.stdout:5500 trunc from 0xbd05c28 to 0x243caa8 2024-04-23T23:50:56.921 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:56.935+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105111 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980197+0000", 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0113423, 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011366052, 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105111 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980197+0000", 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980197+0000", 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980197+0000", 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.11e-07 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980199+0000", 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.265e-06 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980202+0000", 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-06 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980205+0000", 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8229999999999999e-06 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105629 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980761+0000", 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010778322, 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010865681, 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105629 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980761+0000", 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980761+0000", 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980761+0000", 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3699999999999999e-07 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980762+0000", 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.155e-06 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980765+0000", 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0110000000000001e-06 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980769+0000", 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8630000000000001e-06 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105696 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984729+0000", 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0068107339999999997, 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069534590000000004, 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105696 2024-04-23T23:50:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984729+0000", 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984729+0000", 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984729+0000", 2024-04-23T23:50:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984731+0000", 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.339e-06 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984737+0000", 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9540000000000001e-06 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:56.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984742+0000", 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8420000000000004e-06 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105760 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987527+0000", 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0040120650000000004, 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042101420000000001, 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:56.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105760 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987527+0000", 2024-04-23T23:50:56.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987527+0000", 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987528+0000", 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3999999999999998e-07 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987529+0000", 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-06 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987532+0000", 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9380000000000001e-06 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987535+0000", 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.79e-06 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104797 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989691+0000", 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0018481470000000001, 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021002799999999999, 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104797 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989691+0000", 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989691+0000", 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989692+0000", 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989694+0000", 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6989999999999999e-06 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989701+0000", 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6290000000000004e-06 2024-04-23T23:50:57.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989707+0000", 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8379999999999998e-06 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104864 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990771+0000", 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00076862100000000004, 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001075211, 2024-04-23T23:50:57.005 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104864 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990771+0000", 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990771+0000", 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990771+0000", 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2199999999999999e-07 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990774+0000", 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5000000000000002e-06 2024-04-23T23:50:57.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990779+0000", 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4090000000000002e-06 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990784+0000", 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7190000000000001e-06 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105117 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980208+0000", 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011331028999999999, 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011691942, 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105117 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980208+0000", 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980208+0000", 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980209+0000", 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:50:57.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980210+0000", 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.711e-06 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980215+0000", 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9620000000000001e-06 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980218+0000", 2024-04-23T23:50:57.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6220000000000001e-06 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105630 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980792+0000", 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010747049, 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011161858, 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105630 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980792+0000", 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980792+0000", 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980793+0000", 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980794+0000", 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980798+0000", 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.529e-06 2024-04-23T23:50:57.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980801+0000", 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105697 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984749+0000", 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0067901580000000001, 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072686590000000002, 2024-04-23T23:50:57.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105697 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984749+0000", 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984749+0000", 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984750+0000", 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984752+0000", 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:50:57.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984758+0000", 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.285e-06 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984763+0000", 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0119999999999996e-06 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105761 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987538+0000", 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0040012440000000002, 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045370180000000003, 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105761 2024-04-23T23:50:57.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987538+0000", 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987538+0000", 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987538+0000", 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:50:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987540+0000", 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1519999999999999e-06 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987543+0000", 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9079999999999999e-06 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987546+0000", 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7000000000000002e-06 2024-04-23T23:50:57.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104807 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989714+0000", 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001825524, 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024137899999999999, 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104807 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989714+0000", 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989714+0000", 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989714+0000", 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-07 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989716+0000", 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9989999999999998e-06 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989723+0000", 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999999e-06 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989728+0000", 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:50:57.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104865 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990790+0000", 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000749042, 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001389943, 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104865 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990790+0000", 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990790+0000", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990791+0000", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990793+0000", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2520000000000002e-06 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990799+0000", 2024-04-23T23:50:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7540000000000003e-06 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990803+0000", 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4510000000000002e-06 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105119 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980221+0000", 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011318084000000001, 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012016414, 2024-04-23T23:50:57.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105119 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980221+0000", 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980221+0000", 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980222+0000", 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3999999999999998e-07 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980223+0000", 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-04-23T23:50:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980226+0000", 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9830000000000001e-06 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980228+0000", 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6369999999999999e-06 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.032 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105631 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980805+0000", 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010734574, 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011489981, 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105631 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980805+0000", 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980805+0000", 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980805+0000", 2024-04-23T23:50:57.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980807+0000", 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7349999999999999e-06 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980812+0000", 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8999999999999997e-06 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980815+0000", 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9660000000000001e-06 2024-04-23T23:50:57.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105698 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984770+0000", 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0067693659999999998, 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075784349999999997, 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105698 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984770+0000", 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984770+0000", 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984770+0000", 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0400000000000002e-07 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984776+0000", 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259999999999997e-06 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984783+0000", 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6590000000000001e-06 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984788+0000", 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3079999999999998e-06 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105762 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987568+0000", 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0039717770000000001, 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048338060000000004, 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105762 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987568+0000", 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987568+0000", 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987568+0000", 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.82e-07 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987570+0000", 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6050000000000001e-06 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987573+0000", 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.534e-06 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987576+0000", 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9050000000000001e-06 2024-04-23T23:50:57.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104808 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989785+0000", 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0017542930000000001, 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026691879999999999, 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104808 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989785+0000", 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989785+0000", 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989786+0000", 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9699999999999995e-07 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989789+0000", 2024-04-23T23:50:57.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2590000000000001e-06 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989798+0000", 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4999999999999999e-06 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989804+0000", 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6710000000000002e-06 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104866 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990809+0000", 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000730188, 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017020620000000001, 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104866 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990809+0000", 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990809+0000", 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990810+0000", 2024-04-23T23:50:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990812+0000", 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2680000000000001e-06 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990818+0000", 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9499999999999998e-06 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990822+0000", 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5000000000000001e-06 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105632 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980818+0000", 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010721223, 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011747974, 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105632 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980818+0000", 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980818+0000", 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980818+0000", 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980820+0000", 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980824+0000", 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0820000000000001e-06 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980826+0000", 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7180000000000001e-06 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105699 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984814+0000", 2024-04-23T23:50:57.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0067258070000000003, 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078065670000000004, 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105699 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984814+0000", 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984814+0000", 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984814+0000", 2024-04-23T23:50:57.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3399999999999998e-07 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984818+0000", 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629999999999999e-06 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984830+0000", 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1147e-05 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984838+0000", 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.123e-06 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105763 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987580+0000", 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0039597449999999998, 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050931379999999997, 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105763 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987580+0000", 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987580+0000", 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987580+0000", 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987594+0000", 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4532000000000001e-05 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987600+0000", 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7389999999999996e-06 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987604+0000", 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3470000000000001e-06 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104809 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989813+0000", 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00172634, 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029125470000000001, 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104809 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989813+0000", 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989813+0000", 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989814+0000", 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989816+0000", 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989822+0000", 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1029999999999998e-06 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989827+0000", 2024-04-23T23:50:57.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-04-23T23:50:57.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104867 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990840+0000", 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00069905, 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001938008, 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104867 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990840+0000", 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990840+0000", 2024-04-23T23:50:57.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990841+0000", 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2199999999999999e-07 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990843+0000", 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.385e-06 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990849+0000", 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:50:57.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990854+0000", 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105633 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980829+0000", 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010710001, 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012005394000000001, 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105633 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980829+0000", 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980829+0000", 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980830+0000", 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.36e-07 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980831+0000", 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980836+0000", 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1739999999999999e-06 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980839+0000", 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9979999999999999e-06 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.071 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105700 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984849+0000", 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066902380000000003, 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080408589999999992, 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105700 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984849+0000", 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984849+0000", 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984850+0000", 2024-04-23T23:50:57.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-04-23T23:50:57.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984852+0000", 2024-04-23T23:50:57.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0980000000000001e-06 2024-04-23T23:50:57.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984856+0000", 2024-04-23T23:50:57.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2209999999999999e-06 2024-04-23T23:50:57.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984859+0000", 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4790000000000001e-06 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105764 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987611+0000", 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0039280210000000003, 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053315439999999997, 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105764 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987611+0000", 2024-04-23T23:50:57.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987611+0000", 2024-04-23T23:50:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987612+0000", 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5599999999999995e-07 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987614+0000", 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-04-23T23:50:57.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987619+0000", 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987624+0000", 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1479999999999997e-06 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104810 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989837+0000", 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001702084, 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003162907, 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104810 2024-04-23T23:50:57.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989837+0000", 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989837+0000", 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989838+0000", 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.68e-07 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989840+0000", 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989847+0000", 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5810000000000002e-06 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989852+0000", 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4859999999999997e-06 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104876 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990880+0000", 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00065982200000000001, 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021734829999999999, 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104876 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990880+0000", 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990880+0000", 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990880+0000", 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990882+0000", 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990888+0000", 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0530000000000003e-06 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990893+0000", 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105634 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980842+0000", 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010697211, 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012263318, 2024-04-23T23:50:57.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105634 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980842+0000", 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980842+0000", 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980843+0000", 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7e-07 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980844+0000", 2024-04-23T23:50:57.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.248e-06 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980847+0000", 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1310000000000002e-06 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980851+0000", 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9979999999999997e-06 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105701 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984864+0000", 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066758540000000002, 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082898099999999999, 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105701 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984864+0000", 2024-04-23T23:50:57.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984864+0000", 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984864+0000", 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984865+0000", 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.238e-06 2024-04-23T23:50:57.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984868+0000", 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1609999999999999e-06 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984871+0000", 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9979999999999999e-06 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105765 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.101 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987630+0000", 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0039090640000000003, 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055740299999999998, 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105765 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987630+0000", 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987630+0000", 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987631+0000", 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987633+0000", 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0829999999999998e-06 2024-04-23T23:50:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987638+0000", 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1479999999999997e-06 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987643+0000", 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9679999999999997e-06 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104811 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989859+0000", 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001680298, 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033966209999999998, 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104811 2024-04-23T23:50:57.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989859+0000", 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989859+0000", 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989859+0000", 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989862+0000", 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0839999999999999e-06 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989868+0000", 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3099999999999997e-06 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989873+0000", 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104877 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990908+0000", 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00063129600000000005, 2024-04-23T23:50:57.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024003509999999998, 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104877 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990908+0000", 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990908+0000", 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990909+0000", 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000001e-07 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990911+0000", 2024-04-23T23:50:57.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3549999999999999e-06 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990917+0000", 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3980000000000004e-06 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990923+0000", 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9440000000000002e-06 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105636 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.980856+0000", 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010683514999999999, 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012507213999999999, 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105636 2024-04-23T23:50:57.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980856+0000", 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980856+0000", 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980856+0000", 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980858+0000", 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8419999999999999e-06 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980863+0000", 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4079999999999997e-06 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.980869+0000", 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4349999999999996e-06 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105702 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984875+0000", 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066646350000000004, 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085515959999999999, 2024-04-23T23:50:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105702 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984875+0000", 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984875+0000", 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984875+0000", 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5100000000000001e-07 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984876+0000", 2024-04-23T23:50:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1659999999999999e-06 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984879+0000", 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.174e-06 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984882+0000", 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8119999999999999e-06 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105766 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987648+0000", 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003891172, 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058309130000000001, 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105766 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987648+0000", 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987648+0000", 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987649+0000", 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987654+0000", 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6450000000000002e-06 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987659+0000", 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1440000000000002e-06 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987664+0000", 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1960000000000001e-06 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104812 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989880+0000", 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001659537, 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036528720000000002, 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104812 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989880+0000", 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989880+0000", 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989880+0000", 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4499999999999997e-07 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989883+0000", 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.114e-06 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989889+0000", 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3829999999999997e-06 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989894+0000", 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1710000000000001e-06 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104878 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990930+0000", 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00060942099999999996, 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026558049999999998, 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104878 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990930+0000", 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990930+0000", 2024-04-23T23:50:57.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990931+0000", 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3900000000000005e-07 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990932+0000", 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990938+0000", 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9719999999999997e-06 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990943+0000", 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8470000000000003e-06 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105703 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984885+0000", 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066542149999999998, 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087566080000000008, 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105703 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984885+0000", 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984885+0000", 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984885+0000", 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2999999999999999e-07 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984887+0000", 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.105e-06 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984890+0000", 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.066e-06 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984894+0000", 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105767 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987669+0000", 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0038706190000000001, 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006026333, 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105767 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987669+0000", 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987669+0000", 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987669+0000", 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987671+0000", 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.781e-06 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987677+0000", 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9190000000000004e-06 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987681+0000", 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1239999999999998e-06 2024-04-23T23:50:57.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104813 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989943+0000", 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0015962839999999999, 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038052469999999999, 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104813 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989943+0000", 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989943+0000", 2024-04-23T23:50:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989944+0000", 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2699999999999999e-07 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989950+0000", 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.516e-06 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989959+0000", 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6419999999999996e-06 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989973+0000", 2024-04-23T23:50:57.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4134000000000001e-05 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104879 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990950+0000", 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00058944200000000003, 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028513940000000002, 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104879 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990950+0000", 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990950+0000", 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990950+0000", 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2399999999999998e-07 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990953+0000", 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.165e-06 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990958+0000", 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6810000000000003e-06 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990970+0000", 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1756000000000001e-05 2024-04-23T23:50:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105704 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984899+0000", 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066402199999999996, 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089582470000000008, 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105704 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984899+0000", 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984899+0000", 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984900+0000", 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000001e-07 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984901+0000", 2024-04-23T23:50:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9300000000000002e-06 2024-04-23T23:50:57.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984907+0000", 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7640000000000002e-06 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984912+0000", 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6920000000000001e-06 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.154 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105768 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987686+0000", 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0038530470000000001, 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062240919999999996, 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105768 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987686+0000", 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987686+0000", 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987687+0000", 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987689+0000", 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8190000000000001e-06 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987694+0000", 2024-04-23T23:50:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987698+0000", 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0910000000000003e-06 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104814 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989996+0000", 2024-04-23T23:50:57.158 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0015432079999999999, 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003969546, 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104814 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989996+0000", 2024-04-23T23:50:57.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989996+0000", 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989997+0000", 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990000+0000", 2024-04-23T23:50:57.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8009999999999999e-06 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990008+0000", 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9610000000000005e-06 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990014+0000", 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2620000000000004e-06 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104880 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991020+0000", 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00051917399999999996, 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029982709999999998, 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.162 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104880 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991020+0000", 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991020+0000", 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991021+0000", 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4099999999999999e-07 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991023+0000", 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6690000000000002e-06 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991030+0000", 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7089999999999996e-06 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991035+0000", 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1290000000000004e-06 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105705 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984918+0000", 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066209839999999999, 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091554659999999993, 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105705 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984918+0000", 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984918+0000", 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984919+0000", 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.01e-07 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984921+0000", 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2819999999999999e-06 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984927+0000", 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2400000000000004e-06 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984932+0000", 2024-04-23T23:50:57.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-04-23T23:50:57.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.170 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105769 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987702+0000", 2024-04-23T23:50:57.171 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0038371239999999999, 2024-04-23T23:50:57.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064253849999999996, 2024-04-23T23:50:57.171 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105769 2024-04-23T23:50:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987702+0000", 2024-04-23T23:50:57.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987702+0000", 2024-04-23T23:50:57.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987703+0000", 2024-04-23T23:50:57.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2899999999999999e-07 2024-04-23T23:50:57.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987704+0000", 2024-04-23T23:50:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.747e-06 2024-04-23T23:50:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987709+0000", 2024-04-23T23:50:57.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5870000000000003e-06 2024-04-23T23:50:57.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987713+0000", 2024-04-23T23:50:57.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0400000000000003e-06 2024-04-23T23:50:57.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.178 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104824 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990023+0000", 2024-04-23T23:50:57.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00151684, 2024-04-23T23:50:57.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041588170000000004, 2024-04-23T23:50:57.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104824 2024-04-23T23:50:57.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990023+0000", 2024-04-23T23:50:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990023+0000", 2024-04-23T23:50:57.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990023+0000", 2024-04-23T23:50:57.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-07 2024-04-23T23:50:57.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990025+0000", 2024-04-23T23:50:57.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.165e-06 2024-04-23T23:50:57.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990032+0000", 2024-04-23T23:50:57.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5629999999999997e-06 2024-04-23T23:50:57.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990037+0000", 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3970000000000002e-06 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.186 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104881 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991066+0000", 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00047335699999999999, 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031686449999999999, 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104881 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991066+0000", 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991066+0000", 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991067+0000", 2024-04-23T23:50:57.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5799999999999999e-07 2024-04-23T23:50:57.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991069+0000", 2024-04-23T23:50:57.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4679999999999999e-06 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991076+0000", 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4529999999999999e-06 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991081+0000", 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9370000000000003e-06 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105706 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.984938+0000", 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0066011150000000003, 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093529289999999994, 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105706 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984938+0000", 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984938+0000", 2024-04-23T23:50:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984939+0000", 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8799999999999998e-07 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984941+0000", 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2350000000000002e-06 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984947+0000", 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0610000000000001e-06 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.984952+0000", 2024-04-23T23:50:57.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7790000000000003e-06 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105770 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987718+0000", 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0038217049999999999, 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066279540000000001, 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105770 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987718+0000", 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987718+0000", 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987718+0000", 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1100000000000002e-07 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987720+0000", 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8369999999999999e-06 2024-04-23T23:50:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987724+0000", 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5730000000000001e-06 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987728+0000", 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0840000000000002e-06 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104825 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990059+0000", 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0014799889999999999, 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043398229999999996, 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104825 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990059+0000", 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990059+0000", 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990060+0000", 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1099999999999995e-07 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990062+0000", 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.475e-06 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990071+0000", 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8400000000000001e-06 2024-04-23T23:50:57.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990077+0000", 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0519999999999998e-06 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104882 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991101+0000", 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00043830799999999999, 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033512199999999998, 2024-04-23T23:50:57.203 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104882 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991101+0000", 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991101+0000", 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991102+0000", 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991104+0000", 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1040000000000001e-06 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991109+0000", 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.164e-06 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991114+0000", 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6419999999999998e-06 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105771 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987882+0000", 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0036576239999999999, 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066262480000000004, 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105771 2024-04-23T23:50:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987882+0000", 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987882+0000", 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987882+0000", 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6499999999999999e-07 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987886+0000", 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4690000000000002e-06 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987895+0000", 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4390000000000006e-06 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987901+0000", 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8780000000000003e-06 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104826 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990086+0000", 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0014533300000000001, 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004475611, 2024-04-23T23:50:57.210 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104826 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990086+0000", 2024-04-23T23:50:57.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990086+0000", 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990087+0000", 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9599999999999999e-07 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990089+0000", 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.582e-06 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990099+0000", 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8539999999999992e-06 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990104+0000", 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1680000000000003e-06 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104883 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991158+0000", 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00038100299999999998, 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034561140000000001, 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104883 2024-04-23T23:50:57.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991158+0000", 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991158+0000", 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991159+0000", 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7199999999999998e-07 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991167+0000", 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7379999999999994e-06 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991177+0000", 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991184+0000", 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1879999999999996e-06 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105772 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987913+0000", 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0036267750000000001, 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067584280000000004, 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105772 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987913+0000", 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987913+0000", 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987913+0000", 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987916+0000", 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4099999999999998e-06 2024-04-23T23:50:57.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987922+0000", 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7309999999999996e-06 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987927+0000", 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5839999999999996e-06 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104827 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990109+0000", 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001430218, 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046148220000000002, 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104827 2024-04-23T23:50:57.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990109+0000", 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990109+0000", 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990110+0000", 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3599999999999999e-07 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990111+0000", 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5489999999999999e-06 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990115+0000", 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6820000000000001e-06 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990118+0000", 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9610000000000001e-06 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104884 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991196+0000", 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000343889, 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035815109999999999, 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104884 2024-04-23T23:50:57.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991196+0000", 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991196+0000", 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991196+0000", 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0900000000000002e-07 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991199+0000", 2024-04-23T23:50:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5679999999999998e-06 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991206+0000", 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2280000000000001e-06 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991211+0000", 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3240000000000002e-06 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105773 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987934+0000", 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0036050399999999999, 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068975390000000003, 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105773 2024-04-23T23:50:57.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987934+0000", 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987934+0000", 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987935+0000", 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987937+0000", 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5019999999999999e-06 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987945+0000", 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:50:57.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987950+0000", 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1660000000000002e-06 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104828 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990145+0000", 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0013941030000000001, 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047390339999999996, 2024-04-23T23:50:57.232 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104828 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990145+0000", 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990145+0000", 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990146+0000", 2024-04-23T23:50:57.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0100000000000004e-07 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990150+0000", 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3270000000000002e-06 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990163+0000", 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2529e-05 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990171+0000", 2024-04-23T23:50:57.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8800000000000008e-06 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104893 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991219+0000", 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00032030900000000001, 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037172059999999998, 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104893 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991219+0000", 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991219+0000", 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991220+0000", 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7400000000000003e-07 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991222+0000", 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.26e-06 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991229+0000", 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8959999999999997e-06 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991234+0000", 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105775 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987956+0000", 2024-04-23T23:50:57.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0035831249999999999, 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070389509999999999, 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105775 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987956+0000", 2024-04-23T23:50:57.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987956+0000", 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987957+0000", 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987959+0000", 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1710000000000001e-06 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987972+0000", 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2627e-05 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987977+0000", 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2100000000000001e-06 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104829 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990183+0000", 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001356447, 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048657270000000002, 2024-04-23T23:50:57.243 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104829 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990183+0000", 2024-04-23T23:50:57.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990183+0000", 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990184+0000", 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990186+0000", 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5790000000000002e-06 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990193+0000", 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990199+0000", 2024-04-23T23:50:57.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2890000000000003e-06 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104894 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991250+0000", 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000289652, 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003851569, 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104894 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991250+0000", 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991250+0000", 2024-04-23T23:50:57.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991250+0000", 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2699999999999999e-07 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991253+0000", 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.463e-06 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991261+0000", 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9030000000000004e-06 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991266+0000", 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7470000000000002e-06 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:105790 3.17 3.a7ed6c17 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.987984+0000", 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.003555407, 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007171597, 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105790 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987984+0000", 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987984+0000", 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987984+0000", 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987987+0000", 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3999999999999999e-06 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987993+0000", 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1380000000000004e-06 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.987998+0000", 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6550000000000003e-06 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104830 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990206+0000", 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0013334009999999999, 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005002322, 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104830 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990206+0000", 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990206+0000", 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990206+0000", 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6400000000000003e-07 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990209+0000", 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2400000000000002e-06 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990215+0000", 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3509999999999998e-06 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990220+0000", 2024-04-23T23:50:57.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9699999999999998e-06 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104895 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991278+0000", 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00026168099999999999, 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040556680000000001, 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104895 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991278+0000", 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991278+0000", 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991278+0000", 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2600000000000002e-07 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991280+0000", 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991288+0000", 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5739999999999998e-06 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991294+0000", 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5200000000000003e-06 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104763 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 925696~8192 out=8192b] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.261 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.988945+0000", 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0025948400000000002, 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064497410000000002, 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104763 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988945+0000", 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988945+0000", 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988945+0000", 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8899999999999998e-07 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988951+0000", 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988969+0000", 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8527000000000001e-05 2024-04-23T23:50:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988977+0000", 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2039999999999997e-06 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995298+0000", 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006320773 2024-04-23T23:50:57.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995343+0000", 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4657999999999998e-05 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104831 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990242+0000", 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001297512, 2024-04-23T23:50:57.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00522113, 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104831 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990242+0000", 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990242+0000", 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990242+0000", 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990245+0000", 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3769999999999999e-06 2024-04-23T23:50:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990252+0000", 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7270000000000001e-06 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990257+0000", 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8239999999999996e-06 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104896 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991317+0000", 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00022220700000000001, 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041995820000000003, 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104896 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991317+0000", 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991317+0000", 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991318+0000", 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-23T23:50:57.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991320+0000", 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991328+0000", 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9330000000000001e-06 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991334+0000", 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6799999999999998e-06 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104773 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 73728~8192 out=8192b] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.988990+0000", 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.002549338, 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065893280000000002, 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104773 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988990+0000", 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988990+0000", 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988991+0000", 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5599999999999995e-07 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.988993+0000", 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.768e-06 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989001+0000", 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8849999999999999e-06 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989007+0000", 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2199999999999997e-06 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995447+0000", 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064399180000000002 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995530+0000", 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2371000000000001e-05 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104897 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991368+0000", 2024-04-23T23:50:57.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000171514, 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042779640000000004, 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104897 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991368+0000", 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991368+0000", 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991368+0000", 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0999999999999998e-07 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991372+0000", 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3560000000000001e-06 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991381+0000", 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8000000000000004e-06 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991387+0000", 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9200000000000001e-06 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104774 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 196608~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989015+0000", 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.002524292, 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066984610000000002, 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104774 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989015+0000", 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989015+0000", 2024-04-23T23:50:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989016+0000", 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2200000000000004e-07 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989018+0000", 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.368e-06 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989025+0000", 2024-04-23T23:50:57.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6630000000000004e-06 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989031+0000", 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000002e-06 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995639+0000", 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066079099999999998 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995707+0000", 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8413999999999999e-05 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104841 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990303+0000", 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0012362149999999999, 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054750420000000003, 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104841 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990303+0000", 2024-04-23T23:50:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990303+0000", 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990304+0000", 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990306+0000", 2024-04-23T23:50:57.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4200000000000001e-06 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990312+0000", 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3350000000000003e-06 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990318+0000", 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.164e-06 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104898 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991406+0000", 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00013319800000000001, 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044273539999999997, 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104898 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991406+0000", 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991406+0000", 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991407+0000", 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.51e-07 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991409+0000", 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5610000000000001e-06 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991415+0000", 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2790000000000004e-06 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991420+0000", 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8840000000000002e-06 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104775 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 319488~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989046+0000", 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0024931329999999998, 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068468499999999998, 2024-04-23T23:50:57.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104775 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989046+0000", 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989046+0000", 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989047+0000", 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9200000000000001e-07 2024-04-23T23:50:57.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989049+0000", 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5689999999999999e-06 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989057+0000", 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.221e-06 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989063+0000", 2024-04-23T23:50:57.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0739999999999998e-06 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995814+0000", 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067512420000000002 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.995888+0000", 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4080999999999998e-05 2024-04-23T23:50:57.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104842 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990376+0000", 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001163355, 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005584154, 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104842 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990376+0000", 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990376+0000", 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990377+0000", 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3399999999999998e-07 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990383+0000", 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7309999999999998e-06 2024-04-23T23:50:57.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990391+0000", 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5920000000000001e-06 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990397+0000", 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5589999999999996e-06 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104899 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991437+0000", 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00010261500000000001, 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045822529999999997, 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104899 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991437+0000", 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991437+0000", 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991437+0000", 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991439+0000", 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991450+0000", 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0956e-05 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991455+0000", 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8629999999999999e-06 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104776 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 442368~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989071+0000", 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0024684860000000002, 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007010446, 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104776 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989071+0000", 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989071+0000", 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989071+0000", 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989074+0000", 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.057e-06 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989080+0000", 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5239999999999997e-06 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989086+0000", 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0789999999999997e-06 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996014+0000", 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069282950000000001 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996064+0000", 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9700000000000002e-05 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.308 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104843 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990406+0000", 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0011335780000000001, 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057399579999999999, 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104843 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990406+0000", 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990406+0000", 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990406+0000", 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6499999999999999e-07 2024-04-23T23:50:57.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990409+0000", 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1710000000000001e-06 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990415+0000", 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1460000000000001e-06 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990421+0000", 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-04-23T23:50:57.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104900 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991470+0000", 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9356000000000005e-05, 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047300479999999997, 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104900 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991470+0000", 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991470+0000", 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991471+0000", 2024-04-23T23:50:57.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0500000000000004e-07 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991473+0000", 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.232e-06 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991480+0000", 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991485+0000", 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9779999999999996e-06 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104777 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 565248~8192 out=8192b] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989107+0000", 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0024319150000000002, 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071510499999999999, 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104777 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989107+0000", 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989107+0000", 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989108+0000", 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989111+0000", 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-04-23T23:50:57.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989119+0000", 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6200000000000005e-06 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989125+0000", 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9939999999999997e-06 2024-04-23T23:50:57.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996173+0000", 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070478829999999996 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996221+0000", 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8427999999999998e-05 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104844 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990436+0000", 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001103819, 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058914400000000004, 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104844 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990436+0000", 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990436+0000", 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990436+0000", 2024-04-23T23:50:57.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4899999999999995e-07 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990438+0000", 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.34e-06 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990446+0000", 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8120000000000008e-06 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990452+0000", 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2340000000000002e-06 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104901 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991581+0000", 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.999958, 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047991900000000001, 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104901 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991581+0000", 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991581+0000", 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991582+0000", 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8699999999999995e-07 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991585+0000", 2024-04-23T23:50:57.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2329999999999998e-06 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991591+0000", 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8320000000000002e-06 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991595+0000", 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0559999999999998e-06 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.326 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104778 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 688128~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989137+0000", 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0024023669999999999, 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007304624, 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104778 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989137+0000", 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989137+0000", 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989138+0000", 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989140+0000", 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989147+0000", 2024-04-23T23:50:57.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2570000000000001e-06 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989153+0000", 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8850000000000004e-06 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996342+0000", 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071893360000000002 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996423+0000", 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0476999999999995e-05 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104845 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990469+0000", 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0010702979999999999, 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060361649999999996, 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104845 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990469+0000", 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990469+0000", 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990470+0000", 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8400000000000004e-07 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990472+0000", 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2929999999999999e-06 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990479+0000", 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7479999999999996e-06 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990484+0000", 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2739999999999998e-06 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104779 3.17 3:e87fe4e8:::rbd_data.11794a1e34ce.0000000000000004:25 [read 811008~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989172+0000", 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00236696, 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073947459999999998, 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104779 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989172+0000", 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989172+0000", 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989173+0000", 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-04-23T23:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989175+0000", 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.272e-06 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989182+0000", 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0190000000000001e-06 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989188+0000", 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9449999999999998e-06 2024-04-23T23:50:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.996532+0000", 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073442179999999996 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104846 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990501+0000", 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001038289, 2024-04-23T23:50:57.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061271169999999996, 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104846 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990501+0000", 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990501+0000", 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990502+0000", 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990504+0000", 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.396e-06 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990510+0000", 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3439999999999997e-06 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990515+0000", 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0919999999999997e-06 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104780 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989196+0000", 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0023436500000000001, 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074885860000000002, 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104780 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989196+0000", 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989196+0000", 2024-04-23T23:50:57.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989196+0000", 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999995e-07 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989199+0000", 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2869999999999999e-06 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989206+0000", 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5109999999999997e-06 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989212+0000", 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6779999999999997e-06 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104847 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990524+0000", 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001015845, 2024-04-23T23:50:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006211678, 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104847 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990524+0000", 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990524+0000", 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990524+0000", 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:50:57.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990526+0000", 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3250000000000002e-06 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990532+0000", 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990568+0000", 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6029999999999999e-05 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104790 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989302+0000", 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.002237237, 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074941929999999997, 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104790 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989302+0000", 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989302+0000", 2024-04-23T23:50:57.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989303+0000", 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8500000000000001e-07 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989306+0000", 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3179999999999998e-06 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989315+0000", 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0059999999999998e-06 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989322+0000", 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6100000000000002e-06 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104848 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990591+0000", 2024-04-23T23:50:57.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00094886700000000005, 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006260018, 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104848 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990591+0000", 2024-04-23T23:50:57.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990591+0000", 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990591+0000", 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990594+0000", 2024-04-23T23:50:57.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.948e-06 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990602+0000", 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7190000000000001e-06 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990607+0000", 2024-04-23T23:50:57.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4709999999999998e-06 2024-04-23T23:50:57.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.357 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104791 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989333+0000", 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0022068729999999998, 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075755789999999998, 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104791 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989333+0000", 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989333+0000", 2024-04-23T23:50:57.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989333+0000", 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.51e-07 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989335+0000", 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.419e-06 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989343+0000", 2024-04-23T23:50:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1799999999999999e-06 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989348+0000", 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7740000000000001e-06 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104849 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990614+0000", 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00092528599999999999, 2024-04-23T23:50:57.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063526609999999999, 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104849 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990614+0000", 2024-04-23T23:50:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990614+0000", 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990615+0000", 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990617+0000", 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990623+0000", 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9599999999999997e-06 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990627+0000", 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9089999999999999e-06 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104792 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989356+0000", 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0021837599999999999, 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076706930000000001, 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104792 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989356+0000", 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989356+0000", 2024-04-23T23:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989356+0000", 2024-04-23T23:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2200000000000004e-07 2024-04-23T23:50:57.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989362+0000", 2024-04-23T23:50:57.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:50:57.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989369+0000", 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6930000000000001e-06 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989409+0000", 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0617000000000001e-05 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104859 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990634+0000", 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00090550899999999998, 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064458019999999996, 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104859 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990634+0000", 2024-04-23T23:50:57.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990634+0000", 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990634+0000", 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990636+0000", 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0439999999999998e-06 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990642+0000", 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7780000000000004e-06 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990647+0000", 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8040000000000001e-06 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104793 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989435+0000", 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0021044620000000001, 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077020099999999996, 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104793 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989435+0000", 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989435+0000", 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989436+0000", 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3200000000000005e-07 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989439+0000", 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.066e-06 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989447+0000", 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2579999999999995e-06 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989453+0000", 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4899999999999997e-06 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104860 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990653+0000", 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00088602799999999997, 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065378479999999997, 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104860 2024-04-23T23:50:57.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990653+0000", 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990653+0000", 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990654+0000", 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990656+0000", 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1280000000000002e-06 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990662+0000", 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0349999999999998e-06 2024-04-23T23:50:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990667+0000", 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6959999999999996e-06 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104794 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989461+0000", 2024-04-23T23:50:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0020782029999999998, 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077861900000000001, 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104794 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989461+0000", 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989461+0000", 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989462+0000", 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989464+0000", 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989470+0000", 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4819999999999999e-06 2024-04-23T23:50:57.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989530+0000", 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9373000000000002e-05 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104861 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990672+0000", 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00086719499999999997, 2024-04-23T23:50:57.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066278229999999997, 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104861 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990672+0000", 2024-04-23T23:50:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990672+0000", 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990673+0000", 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8699999999999995e-07 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990675+0000", 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1349999999999999e-06 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990681+0000", 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7440000000000004e-06 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990686+0000", 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0440000000000003e-06 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104795 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989636+0000", 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0019034880000000001, 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077160750000000002, 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104795 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989636+0000", 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989636+0000", 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989637+0000", 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0200000000000001e-07 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989640+0000", 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3060000000000002e-06 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989651+0000", 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1049e-05 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989658+0000", 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.99e-06 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104862 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990692+0000", 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00084779899999999999, 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066992249999999996, 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104862 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990692+0000", 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990692+0000", 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990692+0000", 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3300000000000002e-07 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990694+0000", 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9609999999999999e-06 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990700+0000", 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7910000000000001e-06 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990705+0000", 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7960000000000003e-06 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104910 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991938+0000", 2024-04-23T23:50:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.9996014, 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055045629999999996, 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104910 2024-04-23T23:50:57.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991938+0000", 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991938+0000", 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991939+0000", 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1099999999999995e-07 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991941+0000", 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.097e-06 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991947+0000", 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0480000000000004e-06 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991951+0000", 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7709999999999997e-06 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104796 3.17 3.1727fe17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.989668+0000", 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0018712990000000001, 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078065699999999997, 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104796 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989668+0000", 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989668+0000", 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989669+0000", 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989671+0000", 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989678+0000", 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6939999999999998e-06 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.989684+0000", 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6500000000000001e-06 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104863 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.990746+0000", 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00079363100000000005, 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006758773, 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104863 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990746+0000", 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990746+0000", 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990746+0000", 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999995e-07 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990752+0000", 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2279999999999998e-06 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990759+0000", 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1199999999999996e-06 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.990764+0000", 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:104911 3.17 3.a7ed6c17 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:56.991979+0000", 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.9995599, 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055532109999999997, 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104911 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991979+0000", 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991979+0000", 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991980+0000", 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991983+0000", 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7750000000000001e-06 2024-04-23T23:50:57.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991989+0000", 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5849999999999996e-06 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:56.991995+0000", 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2929999999999997e-06 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.415 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:50:57.415 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 108 2024-04-23T23:50:57.415 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:57.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:50:57.417 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-04-23T23:50:57.418 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T23:50:57.418 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:5501 read 0x16ba1f6 thru 0x16c6ac9 (0xc8d4 bytes) 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:5503 punch from 0x31b0a9 to 0x31e886, (0x37dd bytes) 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:5506 punch from 0xf1ebcb to 0xf23007, (0x443c bytes) 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:5507 trunc from 0x243caa8 to 0x1d46c6e 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:5508 trunc from 0x1d46c6e to 0x9c4550d 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:57.419 INFO:teuthology.orchestra.run.smithi092.stdout:5511 write 0xb5cd845 thru 0xb5d3172 (0x592e bytes) 2024-04-23T23:50:57.420 INFO:teuthology.orchestra.run.smithi092.stdout:5512 trunc from 0xb5d3173 to 0xaa37ae7 2024-04-23T23:50:57.420 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:57.420 INFO:teuthology.orchestra.run.smithi092.stdout:5517 clone 37 order 22 su 65536 sc 13 2024-04-23T23:50:57.442 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:57.456+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:57.516 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:57.531+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:57.550 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:57.550 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:57.550 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:57.550 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:57.550 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:57.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:50:57.618 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:57.631+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:57.665 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.862195649, 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:50:57.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:50:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:57.670 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.860151666, 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:50:57.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:50:57.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.858174417999997, 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:50:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.858143785000003, 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:50:57.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.856200934999997, 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:50:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.854115726000003, 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:50:57.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.852106294999999, 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.850206327000002, 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:57.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.846157798999997, 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:50:57.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:50:57.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.844113123, 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:50:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.842039550000003, 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:50:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.840151394000003, 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:50:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:50:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.717 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.840130617, 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:50:57.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.832139122999997, 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.726 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.830204141000003, 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.830178046, 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:50:57.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.448460581999999, 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:50:57.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:57.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.448418162999999, 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:50:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.448382206000002, 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:50:57.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:57.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.448357848000001, 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:57.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:57.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:57.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:57.753 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:57.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:50:57.755 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:50:57.756 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:50:57.757 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-23T23:50:57.369968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1286094688,"num_objects":1637,"num_object_clones":380,"num_object_copies":3274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1637,"num_whiteouts":664,"num_read":125929,"num_read_kb":458176,"num_write":14077,"num_write_kb":245857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":15046498,"num_keys_recovered":126,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12261,"ondisk_log_size":12261,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":556488,"kb_used_data":268048,"kb_used_omap":21,"kb_used_meta":288298,"kb_avail":754418232,"statfs":{"total":773094113280,"available":772524269568,"internally_reserved":0,"allocated":274481152,"data_stored":335360932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":295217525},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":72726146,"num_objects":121,"num_object_clones":19,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":15,"num_read":9040,"num_read_kb":47337,"num_write":1845,"num_write_kb":19621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000598"},"pg_stats":[{"pgid":"3.1f","version":"75'506","reported_seq":8077,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.230977+0000","last_change":"2024-04-23T23:49:53.323402+0000","last_active":"2024-04-23T23:50:55.230977+0000","last_peered":"2024-04-23T23:50:55.230977+0000","last_clean":"2024-04-23T23:50:55.230977+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:50:55.230977+0000","last_undegraded":"2024-04-23T23:50:55.230977+0000","last_fullsized":"2024-04-23T23:50:55.230977+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:40:47.092387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085782299999999996,"stat_sum":{"num_bytes":20642210,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":9571,"num_read_kb":22476,"num_write":673,"num_write_kb":9349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"76'614","reported_seq":3160,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.846388+0000","last_change":"2024-04-23T23:50:36.142638+0000","last_active":"2024-04-23T23:50:55.846388+0000","last_peered":"2024-04-23T23:50:55.846388+0000","last_clean":"2024-04-23T23:50:55.846388+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:50:55.846388+0000","last_undegraded":"2024-04-23T23:50:55.846388+0000","last_fullsized":"2024-04-23T23:50:55.846388+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:35.657134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067565599999999996,"stat_sum":{"num_bytes":115714132,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":4311,"num_read_kb":10624,"num_write":696,"num_write_kb":7025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"75'296","reported_seq":2799,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248982+0000","last_change":"2024-04-23T23:49:53.328510+0000","last_active":"2024-04-23T23:50:54.248982+0000","last_peered":"2024-04-23T23:50:54.248982+0000","last_clean":"2024-04-23T23:50:54.248982+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:50:54.248982+0000","last_undegraded":"2024-04-23T23:50:54.248982+0000","last_fullsized":"2024-04-23T23:50:54.248982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:28.571956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018779,"stat_sum":{"num_bytes":19374214,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":2953,"num_read_kb":8942,"num_write":323,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"75'419","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.343186+0000","last_change":"2024-04-23T23:50:42.176115+0000","last_active":"2024-04-23T23:50:54.343186+0000","last_peered":"2024-04-23T23:50:54.343186+0000","last_clean":"2024-04-23T23:50:54.343186+0000","last_became_active":"2024-04-23T23:50:42.175291+0000","last_became_peered":"2024-04-23T23:50:42.175291+0000","last_unstale":"2024-04-23T23:50:54.343186+0000","last_undegraded":"2024-04-23T23:50:54.343186+0000","last_fullsized":"2024-04-23T23:50:54.343186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:56:34.805561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099769099999999999,"stat_sum":{"num_bytes":61847730,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":4558,"num_read_kb":12953,"num_write":435,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2296821,"num_keys_recovered":67,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'222","reported_seq":3361,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.462290+0000","last_change":"2024-04-23T23:49:53.322810+0000","last_active":"2024-04-23T23:50:55.462290+0000","last_peered":"2024-04-23T23:50:55.462290+0000","last_clean":"2024-04-23T23:50:55.462290+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:50:55.462290+0000","last_undegraded":"2024-04-23T23:50:55.462290+0000","last_fullsized":"2024-04-23T23:50:55.462290+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:19.376065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079418399999999997,"stat_sum":{"num_bytes":53501878,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":3617,"num_read_kb":16996,"num_write":201,"num_write_kb":4951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"75'219","reported_seq":3434,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.352491+0000","last_change":"2024-04-23T23:49:53.322527+0000","last_active":"2024-04-23T23:50:54.352491+0000","last_peered":"2024-04-23T23:50:54.352491+0000","last_clean":"2024-04-23T23:50:54.352491+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:50:54.352491+0000","last_undegraded":"2024-04-23T23:50:54.352491+0000","last_fullsized":"2024-04-23T23:50:54.352491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:38:37.178629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041081400000000001,"stat_sum":{"num_bytes":17137163,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":3611,"num_read_kb":9377,"num_write":234,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"73'683","reported_seq":5694,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.488205+0000","last_change":"2024-04-23T23:50:36.142889+0000","last_active":"2024-04-23T23:50:55.488205+0000","last_peered":"2024-04-23T23:50:55.488205+0000","last_clean":"2024-04-23T23:50:55.488205+0000","last_became_active":"2024-04-23T23:50:36.141286+0000","last_became_peered":"2024-04-23T23:50:36.141286+0000","last_unstale":"2024-04-23T23:50:55.488205+0000","last_undegraded":"2024-04-23T23:50:55.488205+0000","last_fullsized":"2024-04-23T23:50:55.488205+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:08:57.077345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038598300000000001,"stat_sum":{"num_bytes":14432724,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":7726,"num_read_kb":16989,"num_write":978,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":371151,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'516","reported_seq":2633,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.944357+0000","last_change":"2024-04-23T23:50:36.143465+0000","last_active":"2024-04-23T23:50:55.944357+0000","last_peered":"2024-04-23T23:50:55.944357+0000","last_clean":"2024-04-23T23:50:55.944357+0000","last_became_active":"2024-04-23T23:50:36.142717+0000","last_became_peered":"2024-04-23T23:50:36.142717+0000","last_unstale":"2024-04-23T23:50:55.944357+0000","last_undegraded":"2024-04-23T23:50:55.944357+0000","last_fullsized":"2024-04-23T23:50:55.944357+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:25.081417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064860299999999998,"stat_sum":{"num_bytes":42035245,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":3643,"num_read_kb":13644,"num_write":703,"num_write_kb":5420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"75'436","reported_seq":3245,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.553085+0000","last_change":"2024-04-23T23:49:53.323222+0000","last_active":"2024-04-23T23:50:55.553085+0000","last_peered":"2024-04-23T23:50:55.553085+0000","last_clean":"2024-04-23T23:50:55.553085+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:50:55.553085+0000","last_undegraded":"2024-04-23T23:50:55.553085+0000","last_fullsized":"2024-04-23T23:50:55.553085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:04.490304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091814399999999997,"stat_sum":{"num_bytes":70505295,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":3997,"num_read_kb":16739,"num_write":504,"num_write_kb":10490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"73'305","reported_seq":4367,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248965+0000","last_change":"2024-04-23T23:49:53.327873+0000","last_active":"2024-04-23T23:50:54.248965+0000","last_peered":"2024-04-23T23:50:54.248965+0000","last_clean":"2024-04-23T23:50:54.248965+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:50:54.248965+0000","last_undegraded":"2024-04-23T23:50:54.248965+0000","last_fullsized":"2024-04-23T23:50:54.248965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:19:38.647836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039462200000000002,"stat_sum":{"num_bytes":56213470,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":5184,"num_read_kb":29628,"num_write":266,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"73'257","reported_seq":1641,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272481+0000","last_change":"2024-04-23T23:50:33.039363+0000","last_active":"2024-04-23T23:50:54.272481+0000","last_peered":"2024-04-23T23:50:54.272481+0000","last_clean":"2024-04-23T23:50:54.272481+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:50:54.272481+0000","last_undegraded":"2024-04-23T23:50:54.272481+0000","last_fullsized":"2024-04-23T23:50:54.272481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:04.514012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066676300000000001,"stat_sum":{"num_bytes":40983725,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":857,"num_read_kb":12941,"num_write":241,"num_write_kb":12603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"73'197","reported_seq":3021,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.230219+0000","last_change":"2024-04-23T23:49:53.322838+0000","last_active":"2024-04-23T23:50:54.230219+0000","last_peered":"2024-04-23T23:50:54.230219+0000","last_clean":"2024-04-23T23:50:54.230219+0000","last_became_active":"2024-04-23T23:49:51.320365+0000","last_became_peered":"2024-04-23T23:49:51.320365+0000","last_unstale":"2024-04-23T23:50:54.230219+0000","last_undegraded":"2024-04-23T23:50:54.230219+0000","last_fullsized":"2024-04-23T23:50:54.230219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:43:01.701011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042792500000000001,"stat_sum":{"num_bytes":33453348,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":2738,"num_read_kb":12446,"num_write":185,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.239576+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:50:54.239576+0000","last_peered":"2024-04-23T23:50:54.239576+0000","last_clean":"2024-04-23T23:50:54.239576+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:50:54.239576+0000","last_undegraded":"2024-04-23T23:50:54.239576+0000","last_fullsized":"2024-04-23T23:50:54.239576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"76'703","reported_seq":2000,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.485345+0000","last_change":"2024-04-23T23:49:53.322476+0000","last_active":"2024-04-23T23:50:55.485345+0000","last_peered":"2024-04-23T23:50:55.485345+0000","last_clean":"2024-04-23T23:50:55.485345+0000","last_became_active":"2024-04-23T23:49:51.317534+0000","last_became_peered":"2024-04-23T23:49:51.317534+0000","last_unstale":"2024-04-23T23:50:55.485345+0000","last_undegraded":"2024-04-23T23:50:55.485345+0000","last_fullsized":"2024-04-23T23:50:55.485345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:52.378788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046868700000000002,"stat_sum":{"num_bytes":31083996,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":4133,"num_read_kb":10436,"num_write":1075,"num_write_kb":4562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240570+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:50:55.240570+0000","last_peered":"2024-04-23T23:50:55.240570+0000","last_clean":"2024-04-23T23:50:55.240570+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:50:55.240570+0000","last_undegraded":"2024-04-23T23:50:55.240570+0000","last_fullsized":"2024-04-23T23:50:55.240570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'471","reported_seq":3784,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.842062+0000","last_change":"2024-04-23T23:50:36.144636+0000","last_active":"2024-04-23T23:50:55.842062+0000","last_peered":"2024-04-23T23:50:55.842062+0000","last_clean":"2024-04-23T23:50:55.842062+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:50:55.842062+0000","last_undegraded":"2024-04-23T23:50:55.842062+0000","last_fullsized":"2024-04-23T23:50:55.842062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:01.367763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010602859999999999,"stat_sum":{"num_bytes":29256566,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":18,"num_read":5630,"num_read_kb":15587,"num_write":540,"num_write_kb":13958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272521+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:50:54.272521+0000","last_peered":"2024-04-23T23:50:54.272521+0000","last_clean":"2024-04-23T23:50:54.272521+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:50:54.272521+0000","last_undegraded":"2024-04-23T23:50:54.272521+0000","last_fullsized":"2024-04-23T23:50:54.272521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"76'271","reported_seq":1338,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240716+0000","last_change":"2024-04-23T23:49:53.327533+0000","last_active":"2024-04-23T23:50:55.240716+0000","last_peered":"2024-04-23T23:50:55.240716+0000","last_clean":"2024-04-23T23:50:55.240716+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:50:55.240716+0000","last_undegraded":"2024-04-23T23:50:55.240716+0000","last_fullsized":"2024-04-23T23:50:55.240716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:29.328860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056687800000000004,"stat_sum":{"num_bytes":76320585,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":2008,"num_read_kb":7217,"num_write":236,"num_write_kb":3980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240662+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:50:55.240662+0000","last_peered":"2024-04-23T23:50:55.240662+0000","last_clean":"2024-04-23T23:50:55.240662+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:50:55.240662+0000","last_undegraded":"2024-04-23T23:50:55.240662+0000","last_fullsized":"2024-04-23T23:50:55.240662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'373","reported_seq":7215,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.249009+0000","last_change":"2024-04-23T23:50:36.143902+0000","last_active":"2024-04-23T23:50:54.249009+0000","last_peered":"2024-04-23T23:50:54.249009+0000","last_clean":"2024-04-23T23:50:54.249009+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:50:54.249009+0000","last_undegraded":"2024-04-23T23:50:54.249009+0000","last_fullsized":"2024-04-23T23:50:54.249009+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:50.658055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154289,"stat_sum":{"num_bytes":46505600,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":7306,"num_read_kb":29825,"num_write":435,"num_write_kb":11084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"73'269","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.230095+0000","last_change":"2024-04-23T23:49:53.322781+0000","last_active":"2024-04-23T23:50:54.230095+0000","last_peered":"2024-04-23T23:50:54.230095+0000","last_clean":"2024-04-23T23:50:54.230095+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:50:54.230095+0000","last_undegraded":"2024-04-23T23:50:54.230095+0000","last_fullsized":"2024-04-23T23:50:54.230095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:27:36.874190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054288899999999996,"stat_sum":{"num_bytes":11379069,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":2719,"num_read_kb":5992,"num_write":251,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248761+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:50:54.248761+0000","last_peered":"2024-04-23T23:50:54.248761+0000","last_clean":"2024-04-23T23:50:54.248761+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:50:54.248761+0000","last_undegraded":"2024-04-23T23:50:54.248761+0000","last_fullsized":"2024-04-23T23:50:54.248761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:39.497845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"74'235","reported_seq":2059,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240544+0000","last_change":"2024-04-23T23:49:53.327404+0000","last_active":"2024-04-23T23:50:55.240544+0000","last_peered":"2024-04-23T23:50:55.240544+0000","last_clean":"2024-04-23T23:50:55.240544+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:50:55.240544+0000","last_undegraded":"2024-04-23T23:50:55.240544+0000","last_fullsized":"2024-04-23T23:50:55.240544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:25.435265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043824499999999998,"stat_sum":{"num_bytes":23624090,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":2011,"num_read_kb":9958,"num_write":219,"num_write_kb":5813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T23:50:53.232910+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:50:53.232910+0000","last_peered":"2024-04-23T23:50:53.232910+0000","last_clean":"2024-04-23T23:50:53.232910+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:50:53.232910+0000","last_undegraded":"2024-04-23T23:50:53.232910+0000","last_fullsized":"2024-04-23T23:50:53.232910+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"75'260","reported_seq":1427,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.552427+0000","last_change":"2024-04-23T23:49:53.322700+0000","last_active":"2024-04-23T23:50:55.552427+0000","last_peered":"2024-04-23T23:50:55.552427+0000","last_clean":"2024-04-23T23:50:55.552427+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:50:55.552427+0000","last_undegraded":"2024-04-23T23:50:55.552427+0000","last_fullsized":"2024-04-23T23:50:55.552427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:00.656682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085923399999999995,"stat_sum":{"num_bytes":28677294,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":1948,"num_read_kb":6591,"num_write":217,"num_write_kb":2530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272136+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:50:54.272136+0000","last_peered":"2024-04-23T23:50:54.272136+0000","last_clean":"2024-04-23T23:50:54.272136+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:50:54.272136+0000","last_undegraded":"2024-04-23T23:50:54.272136+0000","last_fullsized":"2024-04-23T23:50:54.272136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:28:10.960350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"76'540","reported_seq":1535,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.817584+0000","last_change":"2024-04-23T23:50:33.055605+0000","last_active":"2024-04-23T23:50:55.817584+0000","last_peered":"2024-04-23T23:50:55.817584+0000","last_clean":"2024-04-23T23:50:55.817584+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:50:55.817584+0000","last_undegraded":"2024-04-23T23:50:55.817584+0000","last_fullsized":"2024-04-23T23:50:55.817584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:51.651134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011802519999999999,"stat_sum":{"num_bytes":24578376,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":5649,"num_read_kb":22185,"num_write":744,"num_write_kb":14445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248761+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:50:54.248761+0000","last_peered":"2024-04-23T23:50:54.248761+0000","last_clean":"2024-04-23T23:50:54.248761+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:50:54.248761+0000","last_undegraded":"2024-04-23T23:50:54.248761+0000","last_fullsized":"2024-04-23T23:50:54.248761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272139+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:50:54.272139+0000","last_peered":"2024-04-23T23:50:54.272139+0000","last_clean":"2024-04-23T23:50:54.272139+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:50:54.272139+0000","last_undegraded":"2024-04-23T23:50:54.272139+0000","last_fullsized":"2024-04-23T23:50:54.272139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'253","reported_seq":1113,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.881168+0000","last_change":"2024-04-23T23:50:33.132092+0000","last_active":"2024-04-23T23:50:55.881168+0000","last_peered":"2024-04-23T23:50:55.881168+0000","last_clean":"2024-04-23T23:50:55.881168+0000","last_became_active":"2024-04-23T23:50:32.955054+0000","last_became_peered":"2024-04-23T23:50:32.955054+0000","last_unstale":"2024-04-23T23:50:55.881168+0000","last_undegraded":"2024-04-23T23:50:55.881168+0000","last_fullsized":"2024-04-23T23:50:55.881168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:17.705591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041896000000000001,"stat_sum":{"num_bytes":20394598,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":3995,"num_read_kb":7670,"num_write":242,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":69682,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"74'379","reported_seq":2143,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.268075+0000","last_change":"2024-04-23T23:49:53.322582+0000","last_active":"2024-04-23T23:50:54.268075+0000","last_peered":"2024-04-23T23:50:54.268075+0000","last_clean":"2024-04-23T23:50:54.268075+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:50:54.268075+0000","last_undegraded":"2024-04-23T23:50:54.268075+0000","last_fullsized":"2024-04-23T23:50:54.268075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:11:53.837107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045422700000000003,"stat_sum":{"num_bytes":15212698,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":33,"num_read":3706,"num_read_kb":6620,"num_write":343,"num_write_kb":4083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'314","reported_seq":2842,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.239794+0000","last_change":"2024-04-23T23:49:53.494617+0000","last_active":"2024-04-23T23:50:54.239794+0000","last_peered":"2024-04-23T23:50:54.239794+0000","last_clean":"2024-04-23T23:50:54.239794+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:50:54.239794+0000","last_undegraded":"2024-04-23T23:50:54.239794+0000","last_fullsized":"2024-04-23T23:50:54.239794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:49.547821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039630999999999998,"stat_sum":{"num_bytes":56748874,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":2533,"num_read_kb":11629,"num_write":297,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"76'335","reported_seq":1006,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.941836+0000","last_change":"2024-04-23T23:50:33.091564+0000","last_active":"2024-04-23T23:50:55.941836+0000","last_peered":"2024-04-23T23:50:55.941836+0000","last_clean":"2024-04-23T23:50:55.941836+0000","last_became_active":"2024-04-23T23:50:32.954861+0000","last_became_peered":"2024-04-23T23:50:32.954861+0000","last_unstale":"2024-04-23T23:50:55.941836+0000","last_undegraded":"2024-04-23T23:50:55.941836+0000","last_fullsized":"2024-04-23T23:50:55.941836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:28.328479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001287243,"stat_sum":{"num_bytes":26573090,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":1602,"num_read_kb":9826,"num_write":379,"num_write_kb":6087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":3287017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'333","reported_seq":3375,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248971+0000","last_change":"2024-04-23T23:50:36.143720+0000","last_active":"2024-04-23T23:50:54.248971+0000","last_peered":"2024-04-23T23:50:54.248971+0000","last_clean":"2024-04-23T23:50:54.248971+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:50:54.248971+0000","last_undegraded":"2024-04-23T23:50:54.248971+0000","last_fullsized":"2024-04-23T23:50:54.248971+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:46:01.712916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012607440000000001,"stat_sum":{"num_bytes":39341279,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":3559,"num_read_kb":20497,"num_write":319,"num_write_kb":9757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"73'239","reported_seq":2763,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272447+0000","last_change":"2024-04-23T23:49:53.781509+0000","last_active":"2024-04-23T23:50:54.272447+0000","last_peered":"2024-04-23T23:50:54.272447+0000","last_clean":"2024-04-23T23:50:54.272447+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:50:54.272447+0000","last_undegraded":"2024-04-23T23:50:54.272447+0000","last_fullsized":"2024-04-23T23:50:54.272447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:48.586235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090598199999999995,"stat_sum":{"num_bytes":69041587,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":3061,"num_read_kb":21163,"num_write":235,"num_write_kb":10830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"75'448","reported_seq":2962,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272440+0000","last_change":"2024-04-23T23:49:53.781245+0000","last_active":"2024-04-23T23:50:54.272440+0000","last_peered":"2024-04-23T23:50:54.272440+0000","last_clean":"2024-04-23T23:50:54.272440+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:50:54.272440+0000","last_undegraded":"2024-04-23T23:50:54.272440+0000","last_fullsized":"2024-04-23T23:50:54.272440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.004095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071364900000000001,"stat_sum":{"num_bytes":22565248,"num_objects":61,"num_object_clones":16,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":4107,"num_read_kb":16099,"num_write":615,"num_write_kb":2620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"75'750","reported_seq":3434,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.335804+0000","last_change":"2024-04-23T23:49:53.781604+0000","last_active":"2024-04-23T23:50:54.335804+0000","last_peered":"2024-04-23T23:50:54.335804+0000","last_clean":"2024-04-23T23:50:54.335804+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:50:54.335804+0000","last_undegraded":"2024-04-23T23:50:54.335804+0000","last_fullsized":"2024-04-23T23:50:54.335804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:02:18.712973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097850600000000008,"stat_sum":{"num_bytes":62794537,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":5497,"num_read_kb":21415,"num_write":1142,"num_write_kb":8281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"74'251","reported_seq":4016,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.239611+0000","last_change":"2024-04-23T23:49:53.494621+0000","last_active":"2024-04-23T23:50:54.239611+0000","last_peered":"2024-04-23T23:50:54.239611+0000","last_clean":"2024-04-23T23:50:54.239611+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:50:54.239611+0000","last_undegraded":"2024-04-23T23:50:54.239611+0000","last_fullsized":"2024-04-23T23:50:54.239611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:15.350310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400901,"stat_sum":{"num_bytes":22556505,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":4187,"num_read_kb":10196,"num_write":194,"num_write_kb":3513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"73'314","reported_seq":3654,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272271+0000","last_change":"2024-04-23T23:49:53.780895+0000","last_active":"2024-04-23T23:50:54.272271+0000","last_peered":"2024-04-23T23:50:54.272271+0000","last_clean":"2024-04-23T23:50:54.272271+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:50:54.272271+0000","last_undegraded":"2024-04-23T23:50:54.272271+0000","last_fullsized":"2024-04-23T23:50:54.272271+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:45.644455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049304299999999998,"stat_sum":{"num_bytes":24583847,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":3864,"num_read_kb":10211,"num_write":297,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"75'365","reported_seq":1688,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.488382+0000","last_change":"2024-04-23T23:49:53.323023+0000","last_active":"2024-04-23T23:50:55.488382+0000","last_peered":"2024-04-23T23:50:55.488382+0000","last_clean":"2024-04-23T23:50:55.488382+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:50:55.488382+0000","last_undegraded":"2024-04-23T23:50:55.488382+0000","last_fullsized":"2024-04-23T23:50:55.488382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:27.211119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000902432,"stat_sum":{"num_bytes":66999041,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":16,"num_read":1878,"num_read_kb":16594,"num_write":367,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"73'371","reported_seq":2451,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272267+0000","last_change":"2024-04-23T23:49:53.780967+0000","last_active":"2024-04-23T23:50:54.272267+0000","last_peered":"2024-04-23T23:50:54.272267+0000","last_clean":"2024-04-23T23:50:54.272267+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:50:54.272267+0000","last_undegraded":"2024-04-23T23:50:54.272267+0000","last_fullsized":"2024-04-23T23:50:54.272267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:16.856018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053415600000000004,"stat_sum":{"num_bytes":41426306,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":3654,"num_read_kb":14610,"num_write":332,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1285504320,"num_objects":1635,"num_object_clones":380,"num_object_copies":3270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1635,"num_whiteouts":664,"num_read":125813,"num_read_kb":458076,"num_write":13918,"num_write_kb":243441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":15046498,"num_keys_recovered":126,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":267034624,"data_stored":331677991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":12144,"ondisk_log_size":12144,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168582,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89728,"kb_used_data":52856,"kb_used_omap":10,"kb_used_meta":36853,"kb_avail":94282112,"statfs":{"total":96636764160,"available":96544882688,"internally_reserved":0,"allocated":54124544,"data_stored":64374073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":37737738},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58916,"kb_used_data":19420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94312924,"statfs":{"total":96636764160,"available":96576434176,"internally_reserved":0,"allocated":19886080,"data_stored":22559091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76832,"kb_used_data":33172,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94295008,"statfs":{"total":96636764160,"available":96558088192,"internally_reserved":0,"allocated":33968128,"data_stored":40778212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93544,"kb_used_data":54040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94278296,"statfs":{"total":96636764160,"available":96540975104,"internally_reserved":0,"allocated":55336960,"data_stored":69681833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39720,"kb_used_data":8532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332120,"statfs":{"total":96636764160,"available":96596090880,"internally_reserved":0,"allocated":8736768,"data_stored":9314578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100920,"kb_used_data":61412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94270920,"statfs":{"total":96636764160,"available":96533422080,"internally_reserved":0,"allocated":62885888,"data_stored":79843192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58084,"kb_used_data":31060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94313756,"statfs":{"total":96636764160,"available":96577286144,"internally_reserved":0,"allocated":31805440,"data_stored":38458357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38744,"kb_used_data":7556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333096,"statfs":{"total":96636764160,"available":96597090304,"internally_reserved":0,"allocated":7737344,"data_stored":10351596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32792576,"data_stored":40048762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19283968,"data_stored":22430789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6877184,"data_stored":9959106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53272576,"data_stored":63998539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30949376,"data_stored":38065883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":62013440,"data_stored":79444554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7880704,"data_stored":8922145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53964800,"data_stored":68808213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:50:57.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:50:57.760 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:57.768+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:57.855 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:57.869+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:57.900 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:57.901 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:57.901 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:57.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:50:57.955 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:57.969+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:58.013 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:58.014 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:58.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:50:58.056 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:50:58.070+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:50:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:50:58.089 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:50:58.100 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-23T23:50:57.369968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1286094688,"num_objects":1637,"num_object_clones":380,"num_object_copies":3274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1637,"num_whiteouts":664,"num_read":125929,"num_read_kb":458176,"num_write":14077,"num_write_kb":245857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":15046498,"num_keys_recovered":126,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12261,"ondisk_log_size":12261,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":556488,"kb_used_data":268048,"kb_used_omap":21,"kb_used_meta":288298,"kb_avail":754418232,"statfs":{"total":773094113280,"available":772524269568,"internally_reserved":0,"allocated":274481152,"data_stored":335360932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":295217525},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":72726146,"num_objects":121,"num_object_clones":19,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":15,"num_read":9040,"num_read_kb":47337,"num_write":1845,"num_write_kb":19621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000598"},"pg_stats":[{"pgid":"3.1f","version":"75'506","reported_seq":8077,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.230977+0000","last_change":"2024-04-23T23:49:53.323402+0000","last_active":"2024-04-23T23:50:55.230977+0000","last_peered":"2024-04-23T23:50:55.230977+0000","last_clean":"2024-04-23T23:50:55.230977+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:50:55.230977+0000","last_undegraded":"2024-04-23T23:50:55.230977+0000","last_fullsized":"2024-04-23T23:50:55.230977+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:40:47.092387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085782299999999996,"stat_sum":{"num_bytes":20642210,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":9571,"num_read_kb":22476,"num_write":673,"num_write_kb":9349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"76'614","reported_seq":3160,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.846388+0000","last_change":"2024-04-23T23:50:36.142638+0000","last_active":"2024-04-23T23:50:55.846388+0000","last_peered":"2024-04-23T23:50:55.846388+0000","last_clean":"2024-04-23T23:50:55.846388+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:50:55.846388+0000","last_undegraded":"2024-04-23T23:50:55.846388+0000","last_fullsized":"2024-04-23T23:50:55.846388+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:35.657134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067565599999999996,"stat_sum":{"num_bytes":115714132,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":4311,"num_read_kb":10624,"num_write":696,"num_write_kb":7025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"75'296","reported_seq":2799,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248982+0000","last_change":"2024-04-23T23:49:53.328510+0000","last_active":"2024-04-23T23:50:54.248982+0000","last_peered":"2024-04-23T23:50:54.248982+0000","last_clean":"2024-04-23T23:50:54.248982+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:50:54.248982+0000","last_undegraded":"2024-04-23T23:50:54.248982+0000","last_fullsized":"2024-04-23T23:50:54.248982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:28.571956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018779,"stat_sum":{"num_bytes":19374214,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":2953,"num_read_kb":8942,"num_write":323,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"75'419","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.343186+0000","last_change":"2024-04-23T23:50:42.176115+0000","last_active":"2024-04-23T23:50:54.343186+0000","last_peered":"2024-04-23T23:50:54.343186+0000","last_clean":"2024-04-23T23:50:54.343186+0000","last_became_active":"2024-04-23T23:50:42.175291+0000","last_became_peered":"2024-04-23T23:50:42.175291+0000","last_unstale":"2024-04-23T23:50:54.343186+0000","last_undegraded":"2024-04-23T23:50:54.343186+0000","last_fullsized":"2024-04-23T23:50:54.343186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:56:34.805561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099769099999999999,"stat_sum":{"num_bytes":61847730,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":4558,"num_read_kb":12953,"num_write":435,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2296821,"num_keys_recovered":67,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'222","reported_seq":3361,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.462290+0000","last_change":"2024-04-23T23:49:53.322810+0000","last_active":"2024-04-23T23:50:55.462290+0000","last_peered":"2024-04-23T23:50:55.462290+0000","last_clean":"2024-04-23T23:50:55.462290+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:50:55.462290+0000","last_undegraded":"2024-04-23T23:50:55.462290+0000","last_fullsized":"2024-04-23T23:50:55.462290+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:19.376065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079418399999999997,"stat_sum":{"num_bytes":53501878,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":3617,"num_read_kb":16996,"num_write":201,"num_write_kb":4951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"75'219","reported_seq":3434,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.352491+0000","last_change":"2024-04-23T23:49:53.322527+0000","last_active":"2024-04-23T23:50:54.352491+0000","last_peered":"2024-04-23T23:50:54.352491+0000","last_clean":"2024-04-23T23:50:54.352491+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:50:54.352491+0000","last_undegraded":"2024-04-23T23:50:54.352491+0000","last_fullsized":"2024-04-23T23:50:54.352491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:38:37.178629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041081400000000001,"stat_sum":{"num_bytes":17137163,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":3611,"num_read_kb":9377,"num_write":234,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"73'683","reported_seq":5694,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.488205+0000","last_change":"2024-04-23T23:50:36.142889+0000","last_active":"2024-04-23T23:50:55.488205+0000","last_peered":"2024-04-23T23:50:55.488205+0000","last_clean":"2024-04-23T23:50:55.488205+0000","last_became_active":"2024-04-23T23:50:36.141286+0000","last_became_peered":"2024-04-23T23:50:36.141286+0000","last_unstale":"2024-04-23T23:50:55.488205+0000","last_undegraded":"2024-04-23T23:50:55.488205+0000","last_fullsized":"2024-04-23T23:50:55.488205+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:08:57.077345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038598300000000001,"stat_sum":{"num_bytes":14432724,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":7726,"num_read_kb":16989,"num_write":978,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":371151,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'516","reported_seq":2633,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.944357+0000","last_change":"2024-04-23T23:50:36.143465+0000","last_active":"2024-04-23T23:50:55.944357+0000","last_peered":"2024-04-23T23:50:55.944357+0000","last_clean":"2024-04-23T23:50:55.944357+0000","last_became_active":"2024-04-23T23:50:36.142717+0000","last_became_peered":"2024-04-23T23:50:36.142717+0000","last_unstale":"2024-04-23T23:50:55.944357+0000","last_undegraded":"2024-04-23T23:50:55.944357+0000","last_fullsized":"2024-04-23T23:50:55.944357+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:25.081417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064860299999999998,"stat_sum":{"num_bytes":42035245,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":3643,"num_read_kb":13644,"num_write":703,"num_write_kb":5420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"75'436","reported_seq":3245,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.553085+0000","last_change":"2024-04-23T23:49:53.323222+0000","last_active":"2024-04-23T23:50:55.553085+0000","last_peered":"2024-04-23T23:50:55.553085+0000","last_clean":"2024-04-23T23:50:55.553085+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:50:55.553085+0000","last_undegraded":"2024-04-23T23:50:55.553085+0000","last_fullsized":"2024-04-23T23:50:55.553085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:04.490304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091814399999999997,"stat_sum":{"num_bytes":70505295,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":3997,"num_read_kb":16739,"num_write":504,"num_write_kb":10490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"73'305","reported_seq":4367,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248965+0000","last_change":"2024-04-23T23:49:53.327873+0000","last_active":"2024-04-23T23:50:54.248965+0000","last_peered":"2024-04-23T23:50:54.248965+0000","last_clean":"2024-04-23T23:50:54.248965+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:50:54.248965+0000","last_undegraded":"2024-04-23T23:50:54.248965+0000","last_fullsized":"2024-04-23T23:50:54.248965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:19:38.647836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039462200000000002,"stat_sum":{"num_bytes":56213470,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":5184,"num_read_kb":29628,"num_write":266,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"73'257","reported_seq":1641,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272481+0000","last_change":"2024-04-23T23:50:33.039363+0000","last_active":"2024-04-23T23:50:54.272481+0000","last_peered":"2024-04-23T23:50:54.272481+0000","last_clean":"2024-04-23T23:50:54.272481+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:50:54.272481+0000","last_undegraded":"2024-04-23T23:50:54.272481+0000","last_fullsized":"2024-04-23T23:50:54.272481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:04.514012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066676300000000001,"stat_sum":{"num_bytes":40983725,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":857,"num_read_kb":12941,"num_write":241,"num_write_kb":12603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"73'197","reported_seq":3021,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.230219+0000","last_change":"2024-04-23T23:49:53.322838+0000","last_active":"2024-04-23T23:50:54.230219+0000","last_peered":"2024-04-23T23:50:54.230219+0000","last_clean":"2024-04-23T23:50:54.230219+0000","last_became_active":"2024-04-23T23:49:51.320365+0000","last_became_peered":"2024-04-23T23:49:51.320365+0000","last_unstale":"2024-04-23T23:50:54.230219+0000","last_undegraded":"2024-04-23T23:50:54.230219+0000","last_fullsized":"2024-04-23T23:50:54.230219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:43:01.701011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042792500000000001,"stat_sum":{"num_bytes":33453348,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":2738,"num_read_kb":12446,"num_write":185,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.239576+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:50:54.239576+0000","last_peered":"2024-04-23T23:50:54.239576+0000","last_clean":"2024-04-23T23:50:54.239576+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:50:54.239576+0000","last_undegraded":"2024-04-23T23:50:54.239576+0000","last_fullsized":"2024-04-23T23:50:54.239576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"76'703","reported_seq":2000,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.485345+0000","last_change":"2024-04-23T23:49:53.322476+0000","last_active":"2024-04-23T23:50:55.485345+0000","last_peered":"2024-04-23T23:50:55.485345+0000","last_clean":"2024-04-23T23:50:55.485345+0000","last_became_active":"2024-04-23T23:49:51.317534+0000","last_became_peered":"2024-04-23T23:49:51.317534+0000","last_unstale":"2024-04-23T23:50:55.485345+0000","last_undegraded":"2024-04-23T23:50:55.485345+0000","last_fullsized":"2024-04-23T23:50:55.485345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:52.378788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046868700000000002,"stat_sum":{"num_bytes":31083996,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":4133,"num_read_kb":10436,"num_write":1075,"num_write_kb":4562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240570+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:50:55.240570+0000","last_peered":"2024-04-23T23:50:55.240570+0000","last_clean":"2024-04-23T23:50:55.240570+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:50:55.240570+0000","last_undegraded":"2024-04-23T23:50:55.240570+0000","last_fullsized":"2024-04-23T23:50:55.240570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'471","reported_seq":3784,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.842062+0000","last_change":"2024-04-23T23:50:36.144636+0000","last_active":"2024-04-23T23:50:55.842062+0000","last_peered":"2024-04-23T23:50:55.842062+0000","last_clean":"2024-04-23T23:50:55.842062+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:50:55.842062+0000","last_undegraded":"2024-04-23T23:50:55.842062+0000","last_fullsized":"2024-04-23T23:50:55.842062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:01.367763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010602859999999999,"stat_sum":{"num_bytes":29256566,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":18,"num_read":5630,"num_read_kb":15587,"num_write":540,"num_write_kb":13958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272521+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:50:54.272521+0000","last_peered":"2024-04-23T23:50:54.272521+0000","last_clean":"2024-04-23T23:50:54.272521+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:50:54.272521+0000","last_undegraded":"2024-04-23T23:50:54.272521+0000","last_fullsized":"2024-04-23T23:50:54.272521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"76'271","reported_seq":1338,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240716+0000","last_change":"2024-04-23T23:49:53.327533+0000","last_active":"2024-04-23T23:50:55.240716+0000","last_peered":"2024-04-23T23:50:55.240716+0000","last_clean":"2024-04-23T23:50:55.240716+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:50:55.240716+0000","last_undegraded":"2024-04-23T23:50:55.240716+0000","last_fullsized":"2024-04-23T23:50:55.240716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:29.328860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056687800000000004,"stat_sum":{"num_bytes":76320585,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":2008,"num_read_kb":7217,"num_write":236,"num_write_kb":3980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240662+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:50:55.240662+0000","last_peered":"2024-04-23T23:50:55.240662+0000","last_clean":"2024-04-23T23:50:55.240662+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:50:55.240662+0000","last_undegraded":"2024-04-23T23:50:55.240662+0000","last_fullsized":"2024-04-23T23:50:55.240662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'373","reported_seq":7215,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.249009+0000","last_change":"2024-04-23T23:50:36.143902+0000","last_active":"2024-04-23T23:50:54.249009+0000","last_peered":"2024-04-23T23:50:54.249009+0000","last_clean":"2024-04-23T23:50:54.249009+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:50:54.249009+0000","last_undegraded":"2024-04-23T23:50:54.249009+0000","last_fullsized":"2024-04-23T23:50:54.249009+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:50.658055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154289,"stat_sum":{"num_bytes":46505600,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":7306,"num_read_kb":29825,"num_write":435,"num_write_kb":11084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"73'269","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.230095+0000","last_change":"2024-04-23T23:49:53.322781+0000","last_active":"2024-04-23T23:50:54.230095+0000","last_peered":"2024-04-23T23:50:54.230095+0000","last_clean":"2024-04-23T23:50:54.230095+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:50:54.230095+0000","last_undegraded":"2024-04-23T23:50:54.230095+0000","last_fullsized":"2024-04-23T23:50:54.230095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:27:36.874190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054288899999999996,"stat_sum":{"num_bytes":11379069,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":2719,"num_read_kb":5992,"num_write":251,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248761+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:50:54.248761+0000","last_peered":"2024-04-23T23:50:54.248761+0000","last_clean":"2024-04-23T23:50:54.248761+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:50:54.248761+0000","last_undegraded":"2024-04-23T23:50:54.248761+0000","last_fullsized":"2024-04-23T23:50:54.248761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:39.497845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"74'235","reported_seq":2059,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.240544+0000","last_change":"2024-04-23T23:49:53.327404+0000","last_active":"2024-04-23T23:50:55.240544+0000","last_peered":"2024-04-23T23:50:55.240544+0000","last_clean":"2024-04-23T23:50:55.240544+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:50:55.240544+0000","last_undegraded":"2024-04-23T23:50:55.240544+0000","last_fullsized":"2024-04-23T23:50:55.240544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:25.435265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043824499999999998,"stat_sum":{"num_bytes":23624090,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":2011,"num_read_kb":9958,"num_write":219,"num_write_kb":5813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T23:50:53.232910+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:50:53.232910+0000","last_peered":"2024-04-23T23:50:53.232910+0000","last_clean":"2024-04-23T23:50:53.232910+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:50:53.232910+0000","last_undegraded":"2024-04-23T23:50:53.232910+0000","last_fullsized":"2024-04-23T23:50:53.232910+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"75'260","reported_seq":1427,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.552427+0000","last_change":"2024-04-23T23:49:53.322700+0000","last_active":"2024-04-23T23:50:55.552427+0000","last_peered":"2024-04-23T23:50:55.552427+0000","last_clean":"2024-04-23T23:50:55.552427+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:50:55.552427+0000","last_undegraded":"2024-04-23T23:50:55.552427+0000","last_fullsized":"2024-04-23T23:50:55.552427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:00.656682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085923399999999995,"stat_sum":{"num_bytes":28677294,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":1948,"num_read_kb":6591,"num_write":217,"num_write_kb":2530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272136+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:50:54.272136+0000","last_peered":"2024-04-23T23:50:54.272136+0000","last_clean":"2024-04-23T23:50:54.272136+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:50:54.272136+0000","last_undegraded":"2024-04-23T23:50:54.272136+0000","last_fullsized":"2024-04-23T23:50:54.272136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:28:10.960350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"76'540","reported_seq":1535,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.817584+0000","last_change":"2024-04-23T23:50:33.055605+0000","last_active":"2024-04-23T23:50:55.817584+0000","last_peered":"2024-04-23T23:50:55.817584+0000","last_clean":"2024-04-23T23:50:55.817584+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:50:55.817584+0000","last_undegraded":"2024-04-23T23:50:55.817584+0000","last_fullsized":"2024-04-23T23:50:55.817584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:51.651134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011802519999999999,"stat_sum":{"num_bytes":24578376,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":5649,"num_read_kb":22185,"num_write":744,"num_write_kb":14445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248761+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:50:54.248761+0000","last_peered":"2024-04-23T23:50:54.248761+0000","last_clean":"2024-04-23T23:50:54.248761+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:50:54.248761+0000","last_undegraded":"2024-04-23T23:50:54.248761+0000","last_fullsized":"2024-04-23T23:50:54.248761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272139+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:50:54.272139+0000","last_peered":"2024-04-23T23:50:54.272139+0000","last_clean":"2024-04-23T23:50:54.272139+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:50:54.272139+0000","last_undegraded":"2024-04-23T23:50:54.272139+0000","last_fullsized":"2024-04-23T23:50:54.272139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'253","reported_seq":1113,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.881168+0000","last_change":"2024-04-23T23:50:33.132092+0000","last_active":"2024-04-23T23:50:55.881168+0000","last_peered":"2024-04-23T23:50:55.881168+0000","last_clean":"2024-04-23T23:50:55.881168+0000","last_became_active":"2024-04-23T23:50:32.955054+0000","last_became_peered":"2024-04-23T23:50:32.955054+0000","last_unstale":"2024-04-23T23:50:55.881168+0000","last_undegraded":"2024-04-23T23:50:55.881168+0000","last_fullsized":"2024-04-23T23:50:55.881168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:17.705591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041896000000000001,"stat_sum":{"num_bytes":20394598,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":3995,"num_read_kb":7670,"num_write":242,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":69682,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"74'379","reported_seq":2143,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.268075+0000","last_change":"2024-04-23T23:49:53.322582+0000","last_active":"2024-04-23T23:50:54.268075+0000","last_peered":"2024-04-23T23:50:54.268075+0000","last_clean":"2024-04-23T23:50:54.268075+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:50:54.268075+0000","last_undegraded":"2024-04-23T23:50:54.268075+0000","last_fullsized":"2024-04-23T23:50:54.268075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:11:53.837107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045422700000000003,"stat_sum":{"num_bytes":15212698,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":33,"num_read":3706,"num_read_kb":6620,"num_write":343,"num_write_kb":4083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'314","reported_seq":2842,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.239794+0000","last_change":"2024-04-23T23:49:53.494617+0000","last_active":"2024-04-23T23:50:54.239794+0000","last_peered":"2024-04-23T23:50:54.239794+0000","last_clean":"2024-04-23T23:50:54.239794+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:50:54.239794+0000","last_undegraded":"2024-04-23T23:50:54.239794+0000","last_fullsized":"2024-04-23T23:50:54.239794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:49.547821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039630999999999998,"stat_sum":{"num_bytes":56748874,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":2533,"num_read_kb":11629,"num_write":297,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"76'335","reported_seq":1006,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.941836+0000","last_change":"2024-04-23T23:50:33.091564+0000","last_active":"2024-04-23T23:50:55.941836+0000","last_peered":"2024-04-23T23:50:55.941836+0000","last_clean":"2024-04-23T23:50:55.941836+0000","last_became_active":"2024-04-23T23:50:32.954861+0000","last_became_peered":"2024-04-23T23:50:32.954861+0000","last_unstale":"2024-04-23T23:50:55.941836+0000","last_undegraded":"2024-04-23T23:50:55.941836+0000","last_fullsized":"2024-04-23T23:50:55.941836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:28.328479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001287243,"stat_sum":{"num_bytes":26573090,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":1602,"num_read_kb":9826,"num_write":379,"num_write_kb":6087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":3287017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'333","reported_seq":3375,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.248971+0000","last_change":"2024-04-23T23:50:36.143720+0000","last_active":"2024-04-23T23:50:54.248971+0000","last_peered":"2024-04-23T23:50:54.248971+0000","last_clean":"2024-04-23T23:50:54.248971+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:50:54.248971+0000","last_undegraded":"2024-04-23T23:50:54.248971+0000","last_fullsized":"2024-04-23T23:50:54.248971+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:46:01.712916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012607440000000001,"stat_sum":{"num_bytes":39341279,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":3559,"num_read_kb":20497,"num_write":319,"num_write_kb":9757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"73'239","reported_seq":2763,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272447+0000","last_change":"2024-04-23T23:49:53.781509+0000","last_active":"2024-04-23T23:50:54.272447+0000","last_peered":"2024-04-23T23:50:54.272447+0000","last_clean":"2024-04-23T23:50:54.272447+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:50:54.272447+0000","last_undegraded":"2024-04-23T23:50:54.272447+0000","last_fullsized":"2024-04-23T23:50:54.272447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:48.586235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090598199999999995,"stat_sum":{"num_bytes":69041587,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":3061,"num_read_kb":21163,"num_write":235,"num_write_kb":10830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"75'448","reported_seq":2962,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272440+0000","last_change":"2024-04-23T23:49:53.781245+0000","last_active":"2024-04-23T23:50:54.272440+0000","last_peered":"2024-04-23T23:50:54.272440+0000","last_clean":"2024-04-23T23:50:54.272440+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:50:54.272440+0000","last_undegraded":"2024-04-23T23:50:54.272440+0000","last_fullsized":"2024-04-23T23:50:54.272440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.004095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071364900000000001,"stat_sum":{"num_bytes":22565248,"num_objects":61,"num_object_clones":16,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":4107,"num_read_kb":16099,"num_write":615,"num_write_kb":2620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"75'750","reported_seq":3434,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.335804+0000","last_change":"2024-04-23T23:49:53.781604+0000","last_active":"2024-04-23T23:50:54.335804+0000","last_peered":"2024-04-23T23:50:54.335804+0000","last_clean":"2024-04-23T23:50:54.335804+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:50:54.335804+0000","last_undegraded":"2024-04-23T23:50:54.335804+0000","last_fullsized":"2024-04-23T23:50:54.335804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:02:18.712973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097850600000000008,"stat_sum":{"num_bytes":62794537,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":5497,"num_read_kb":21415,"num_write":1142,"num_write_kb":8281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"74'251","reported_seq":4016,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.239611+0000","last_change":"2024-04-23T23:49:53.494621+0000","last_active":"2024-04-23T23:50:54.239611+0000","last_peered":"2024-04-23T23:50:54.239611+0000","last_clean":"2024-04-23T23:50:54.239611+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:50:54.239611+0000","last_undegraded":"2024-04-23T23:50:54.239611+0000","last_fullsized":"2024-04-23T23:50:54.239611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:15.350310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400901,"stat_sum":{"num_bytes":22556505,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":4187,"num_read_kb":10196,"num_write":194,"num_write_kb":3513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"73'314","reported_seq":3654,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272271+0000","last_change":"2024-04-23T23:49:53.780895+0000","last_active":"2024-04-23T23:50:54.272271+0000","last_peered":"2024-04-23T23:50:54.272271+0000","last_clean":"2024-04-23T23:50:54.272271+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:50:54.272271+0000","last_undegraded":"2024-04-23T23:50:54.272271+0000","last_fullsized":"2024-04-23T23:50:54.272271+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:45.644455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049304299999999998,"stat_sum":{"num_bytes":24583847,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":3864,"num_read_kb":10211,"num_write":297,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"75'365","reported_seq":1688,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T23:50:55.488382+0000","last_change":"2024-04-23T23:49:53.323023+0000","last_active":"2024-04-23T23:50:55.488382+0000","last_peered":"2024-04-23T23:50:55.488382+0000","last_clean":"2024-04-23T23:50:55.488382+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:50:55.488382+0000","last_undegraded":"2024-04-23T23:50:55.488382+0000","last_fullsized":"2024-04-23T23:50:55.488382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:27.211119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000902432,"stat_sum":{"num_bytes":66999041,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":16,"num_read":1878,"num_read_kb":16594,"num_write":367,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"73'371","reported_seq":2451,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-23T23:50:54.272267+0000","last_change":"2024-04-23T23:49:53.780967+0000","last_active":"2024-04-23T23:50:54.272267+0000","last_peered":"2024-04-23T23:50:54.272267+0000","last_clean":"2024-04-23T23:50:54.272267+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:50:54.272267+0000","last_undegraded":"2024-04-23T23:50:54.272267+0000","last_fullsized":"2024-04-23T23:50:54.272267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:16.856018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053415600000000004,"stat_sum":{"num_bytes":41426306,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":3654,"num_read_kb":14610,"num_write":332,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1285504320,"num_objects":1635,"num_object_clones":380,"num_object_copies":3270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1635,"num_whiteouts":664,"num_read":125813,"num_read_kb":458076,"num_write":13918,"num_write_kb":243441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":15046498,"num_keys_recovered":126,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":267034624,"data_stored":331677991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":12144,"ondisk_log_size":12144,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168582,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89728,"kb_used_data":52856,"kb_used_omap":10,"kb_used_meta":36853,"kb_avail":94282112,"statfs":{"total":96636764160,"available":96544882688,"internally_reserved":0,"allocated":54124544,"data_stored":64374073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":37737738},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58916,"kb_used_data":19420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94312924,"statfs":{"total":96636764160,"available":96576434176,"internally_reserved":0,"allocated":19886080,"data_stored":22559091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76832,"kb_used_data":33172,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94295008,"statfs":{"total":96636764160,"available":96558088192,"internally_reserved":0,"allocated":33968128,"data_stored":40778212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93544,"kb_used_data":54040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94278296,"statfs":{"total":96636764160,"available":96540975104,"internally_reserved":0,"allocated":55336960,"data_stored":69681833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39720,"kb_used_data":8532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332120,"statfs":{"total":96636764160,"available":96596090880,"internally_reserved":0,"allocated":8736768,"data_stored":9314578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100920,"kb_used_data":61412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94270920,"statfs":{"total":96636764160,"available":96533422080,"internally_reserved":0,"allocated":62885888,"data_stored":79843192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58084,"kb_used_data":31060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94313756,"statfs":{"total":96636764160,"available":96577286144,"internally_reserved":0,"allocated":31805440,"data_stored":38458357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38744,"kb_used_data":7556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333096,"statfs":{"total":96636764160,"available":96597090304,"internally_reserved":0,"allocated":7737344,"data_stored":10351596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32792576,"data_stored":40048762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19283968,"data_stored":22430789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6877184,"data_stored":9959106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53272576,"data_stored":63998539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30949376,"data_stored":38065883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":62013440,"data_stored":79444554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7880704,"data_stored":8922145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53964800,"data_stored":68808213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:50:58.102 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T23:50:58.103 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:50:58.103 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T23:50:58.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T23:50:58.129 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:58.129 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:50:58.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:50:58.129 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:50:58.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.628871294, 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.628405307000001, 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:50:58.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.627898758000001, 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.138 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:50:58.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.627602850000002, 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:50:58.142 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:50:58.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.627415837000001, 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:50:58.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.627214166999998, 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.627073905000003, 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:50:58.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.626828170000003, 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:50:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:50:58.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:50:58.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.626550645000002, 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:50:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:50:58.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.626308072999997, 2024-04-23T23:50:58.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:50:58.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.172 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.626129708999997, 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:50:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:50:58.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.625847118000003, 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.625654634, 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.181 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:50:58.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:50:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.625226693000002, 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:50:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.625074359999999, 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:58.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.624971277, 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:50:58.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:50:58.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.624654331000002, 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.624412904000003, 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:50:58.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:50:58.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.624188357000001, 2024-04-23T23:50:58.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.210 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.623933460000003, 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:50:58.215 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:50:58.215 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:58.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:50:58.217 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:58.230+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:58.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:58.237 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:58.237 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T23:50:58.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:50:58.316 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:58.330+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:58.335 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone36 from 0xaa37ae7 (overlap 0x1d46c6e) to 0x5daf49 2024-04-23T23:50:58.360 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:58.360 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:58.360 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:50:58.360 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:58.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:50:58.417 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:58.431+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:58.437 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:58.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:58.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:50:58.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:58.442 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:58.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T23:50:58.457 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-04-23T23:50:58.518 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:58.532+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:58.578 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:50:58.578 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:50:58.578 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:50:58.578 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:50:58.578 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:50:58.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:50:58.638 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:58.652+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:58.658 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:50:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:50:58.670 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:58.670 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:58.670 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:58.671 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:58.673 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:58.673 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:58.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T23:50:58.693 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:50:58.694 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:50:58.711 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:50:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:50:58.719 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:58.733+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:58.730 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:50:58.731 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:50:58.734 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:58.734 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:58.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:50:58.748 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:50:58.748 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:50:58.766 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:50:58.766 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:50:58.783 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:50:58.783 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:50:58.803 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:50:58.803 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:50:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:58.849 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:58.861+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:58.849 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:58.850 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:58.851 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:58.852 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:58.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-04-23T23:50:58.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:50:58.936 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:50:58.950+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:50:58.939 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:58.939 INFO:teuthology.orchestra.run.smithi092.stdout:5519 read 0x2dd66e4 thru 0x2dd9165 (0x2a82 bytes) 2024-04-23T23:50:58.951 INFO:teuthology.orchestra.run.smithi092.stdout:5520 trunc from 0xaa37ae7 to 0x16525d8 2024-04-23T23:50:58.966 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:58.966 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:58.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:50:59.025 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:59.039+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:59.053 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:59.053 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:59.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-04-23T23:50:59.071 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.071 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.092 INFO:teuthology.orchestra.run.smithi092.stdout:5523 write 0xe361f11 thru 0xe369c0f (0x7cff bytes) 2024-04-23T23:50:59.095 INFO:teuthology.orchestra.run.smithi092.stdout:5524 read 0x931226f thru 0x93211ab (0xef3d bytes) 2024-04-23T23:50:59.095 INFO:teuthology.orchestra.run.smithi092.stdout:5525 read 0x4088b82 thru 0x4098067 (0xf4e6 bytes) 2024-04-23T23:50:59.095 INFO:teuthology.orchestra.run.smithi092.stdout:5526 write 0x10f91a7 thru 0x10fa2c4 (0x111e bytes) 2024-04-23T23:50:59.095 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.095 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.096 INFO:teuthology.orchestra.run.smithi092.stdout:5528 writesame 0xddf320f thru 0xddff172 data_size 0x12(0xbf64 bytes) 2024-04-23T23:50:59.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:50:59.114 INFO:teuthology.orchestra.run.smithi092.stdout:5530 trunc from 0xe369c10 to 0x2f45805 2024-04-23T23:50:59.122 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:59.136+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:59.127 INFO:teuthology.orchestra.run.smithi092.stdout:5531 punch from 0x254cde to 0x25b67c, (0x699e bytes) 2024-04-23T23:50:59.127 INFO:teuthology.orchestra.run.smithi092.stdout:5533 read 0x114b48c thru 0x114de27 (0x299c bytes) 2024-04-23T23:50:59.133 INFO:teuthology.orchestra.run.smithi092.stdout:5534 writesame 0x3111bb8 thru 0x311a06b data_size 0x26(0x84b4 bytes) 2024-04-23T23:50:59.137 INFO:teuthology.orchestra.run.smithi092.stdout:5535 trunc from 0x311a06c to 0x88b24c 2024-04-23T23:50:59.150 INFO:teuthology.orchestra.run.smithi092.stdout:5536 punch from 0x41be5d to 0x42a036, (0xe1d9 bytes) 2024-04-23T23:50:59.150 INFO:teuthology.orchestra.run.smithi092.stdout:5537 read 0x3fea71 thru 0x40b3e5 (0xc975 bytes) 2024-04-23T23:50:59.151 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.151 INFO:teuthology.orchestra.run.smithi092.stdout:5541 punch from 0x1f1cde to 0x1f24ea, (0x80c bytes) 2024-04-23T23:50:59.151 INFO:teuthology.orchestra.run.smithi092.stdout:5542 read 0x35ee34 thru 0x363606 (0x47d3 bytes) 2024-04-23T23:50:59.163 INFO:teuthology.orchestra.run.smithi092.stdout:5543 write 0x602c634 thru 0x6038afe (0xc4cb bytes) 2024-04-23T23:50:59.170 INFO:teuthology.orchestra.run.smithi092.stdout:5544 read 0x15bd4ee thru 0x15c9615 (0xc128 bytes) 2024-04-23T23:50:59.170 INFO:teuthology.orchestra.run.smithi092.stdout:5545 writesame 0x14a099f thru 0x14a4b55 data_size 0x1(0x41b7 bytes) 2024-04-23T23:50:59.197 INFO:teuthology.orchestra.run.smithi092.stdout:5546 write 0xed41ae3 thru 0xed51828 (0xfd46 bytes) 2024-04-23T23:50:59.202 INFO:teuthology.orchestra.run.smithi092.stdout:5547 read 0x4948c2e thru 0x494dddf (0x51b2 bytes) 2024-04-23T23:50:59.203 INFO:teuthology.orchestra.run.smithi092.stdout:5548 write 0xc9b9da thru 0xca61b3 (0xa7da bytes) 2024-04-23T23:50:59.204 INFO:teuthology.orchestra.run.smithi092.stdout:5549 punch from 0x1ff01b5 to 0x1ff4ebd, (0x4d08 bytes) 2024-04-23T23:50:59.217 INFO:teuthology.orchestra.run.smithi092.stdout:5550 trunc from 0xed51829 to 0x43d649b 2024-04-23T23:50:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:59.224 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:59.237+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:59.226 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:59.227 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:59.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-04-23T23:50:59.235 INFO:teuthology.orchestra.run.smithi092.stdout:5551 write 0x4dcb001 thru 0x4dcb156 (0x156 bytes) 2024-04-23T23:50:59.254 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.254 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:50:59.256 INFO:teuthology.orchestra.run.smithi092.stdout:5552 write 0xc508dd7 thru 0xc50e0b0 (0x52da bytes) 2024-04-23T23:50:59.263 INFO:teuthology.orchestra.run.smithi092.stdout:5553 read 0x1f7a765 thru 0x1f84f0f (0xa7ab bytes) 2024-04-23T23:50:59.263 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.263 INFO:teuthology.orchestra.run.smithi092.stdout:5556 write 0x6e3a452 thru 0x6e4638e (0xbf3d bytes) 2024-04-23T23:50:59.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:50:59.269 INFO:teuthology.orchestra.run.smithi092.stdout:5557 write 0x707854d thru 0x7080c8b (0x873f bytes) 2024-04-23T23:50:59.273 INFO:teuthology.orchestra.run.smithi092.stdout:5558 write 0x1c2582c thru 0x1c27b01 (0x22d6 bytes) 2024-04-23T23:50:59.276 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.276 INFO:teuthology.orchestra.run.smithi092.stdout:5560 read 0x57e8041 thru 0x57f3f01 (0xbec1 bytes) 2024-04-23T23:50:59.276 INFO:teuthology.orchestra.run.smithi092.stdout:5561 punch from 0x72a3889 to 0x72acf92, (0x9709 bytes) 2024-04-23T23:50:59.276 INFO:teuthology.orchestra.run.smithi092.stdout:5562 write 0x2bc9b6a thru 0x2bd3d29 (0xa1c0 bytes) 2024-04-23T23:50:59.294 INFO:teuthology.orchestra.run.smithi092.stdout:5563 trunc from 0xc50e0b1 to 0x14ecfa0 2024-04-23T23:50:59.313 INFO:teuthology.orchestra.run.smithi092.stdout:5564 read 0x25c55c thru 0x262d1a (0x67bf bytes) 2024-04-23T23:50:59.314 INFO:teuthology.orchestra.run.smithi092.stdout:5565 read 0xdde9cc thru 0xde6827 (0x7e5c bytes) 2024-04-23T23:50:59.315 INFO:teuthology.orchestra.run.smithi092.stdout:5566 trunc from 0x14ecfa0 to 0x259023c 2024-04-23T23:50:59.328 INFO:teuthology.orchestra.run.smithi092.stdout:5568 write 0x461a827 thru 0x461ca2f (0x2209 bytes) 2024-04-23T23:50:59.331 INFO:teuthology.orchestra.run.smithi092.stdout:5571 write 0x1435957 thru 0x143a893 (0x4f3d bytes) 2024-04-23T23:50:59.333 INFO:teuthology.orchestra.run.smithi092.stdout:5572 write 0x137b7aa thru 0x1384445 (0x8c9c bytes) 2024-04-23T23:50:59.335 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.342 INFO:teuthology.orchestra.run.smithi092.stdout:5577 trunc from 0x461ca30 to 0x9a09200 2024-04-23T23:50:59.347 INFO:teuthology.orchestra.run.smithi092.stdout:5578 punch from 0x858120d to 0x858cdd9, (0xbbcc bytes) 2024-04-23T23:50:59.347 INFO:teuthology.orchestra.run.smithi092.stdout:5580 read 0x1a6838e thru 0x1a76484 (0xe0f7 bytes) 2024-04-23T23:50:59.347 INFO:teuthology.orchestra.run.smithi092.stdout:5581 punch from 0x53bb3da to 0x53c759c, (0xc1c2 bytes) 2024-04-23T23:50:59.349 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:50:59.363+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:50:59.358 INFO:teuthology.orchestra.run.smithi092.stdout:5584 write 0xd1e15f3 thru 0xd1ea605 (0x9013 bytes) 2024-04-23T23:50:59.367 INFO:teuthology.orchestra.run.smithi092.stdout:5585 write 0xd9af715 thru 0xd9b6c27 (0x7513 bytes) 2024-04-23T23:50:59.371 INFO:teuthology.orchestra.run.smithi092.stdout:5586 write 0x7f84bf4 thru 0x7f88afb (0x3f08 bytes) 2024-04-23T23:50:59.373 INFO:teuthology.orchestra.run.smithi092.stdout:5587 read 0x68026d6 thru 0x680608b (0x39b6 bytes) 2024-04-23T23:50:59.375 INFO:teuthology.orchestra.run.smithi092.stdout:5588 trunc from 0xd9b6c28 to 0xe70be88 2024-04-23T23:50:59.380 INFO:teuthology.orchestra.run.smithi092.stdout:5589 write 0xc008330 thru 0xc0151f1 (0xcec2 bytes) 2024-04-23T23:50:59.385 INFO:teuthology.orchestra.run.smithi092.stdout:5593 read 0x75ec892 thru 0x75f1efe (0x566d bytes) 2024-04-23T23:50:59.385 INFO:teuthology.orchestra.run.smithi092.stdout:5595 write 0xcffd8e4 thru 0xd0081f0 (0xa90d bytes) 2024-04-23T23:50:59.388 INFO:teuthology.orchestra.run.smithi092.stdout:5596 read 0x6358b44 thru 0x636738c (0xe849 bytes) 2024-04-23T23:50:59.388 INFO:teuthology.orchestra.run.smithi092.stdout:5597 write 0xd2c2225 thru 0xd2ccb77 (0xa953 bytes) 2024-04-23T23:50:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stdout:5598 punch from 0x7a3f201 to 0x7a453ca, (0x61c9 bytes) 2024-04-23T23:50:59.392 INFO:teuthology.orchestra.run.smithi092.stdout:5600 write 0xc0aedd2 thru 0xc0b6c03 (0x7e32 bytes) 2024-04-23T23:50:59.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:59.395 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:59.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-04-23T23:50:59.396 INFO:teuthology.orchestra.run.smithi092.stdout:5601 read 0x931b4b thru 0x93e345 (0xc7fb bytes) 2024-04-23T23:50:59.396 INFO:teuthology.orchestra.run.smithi092.stdout:5603 writesame 0x4770b39 thru 0x4772edd data_size 0x19(0x23a5 bytes) 2024-04-23T23:50:59.398 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.398 INFO:teuthology.orchestra.run.smithi092.stdout:5609 write 0x6748531 thru 0x6757e55 (0xf925 bytes) 2024-04-23T23:50:59.402 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.402 INFO:teuthology.orchestra.run.smithi092.stdout:5611 write 0xc9ba474 thru 0xc9c5056 (0xabe3 bytes) 2024-04-23T23:50:59.405 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.405 INFO:teuthology.orchestra.run.smithi092.stdout:5615 write 0x2376424 thru 0x2386323 (0xff00 bytes) 2024-04-23T23:50:59.407 INFO:teuthology.orchestra.run.smithi092.stdout:5616 punch from 0xd832052 to 0xd83fc8f, (0xdc3d bytes) 2024-04-23T23:50:59.407 INFO:teuthology.orchestra.run.smithi092.stdout:5617 write 0x219012a thru 0x2191acf (0x19a6 bytes) 2024-04-23T23:50:59.408 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.408 INFO:teuthology.orchestra.run.smithi092.stdout:5620 write 0x695d554 thru 0x6960f4c (0x39f9 bytes) 2024-04-23T23:50:59.411 INFO:teuthology.orchestra.run.smithi092.stdout:5621 writesame 0x2054ee8 thru 0x206423f data_size 0x34(0xf358 bytes) 2024-04-23T23:50:59.413 INFO:teuthology.orchestra.run.smithi092.stdout:5622 write 0x15fc9f5 thru 0x1606c4f (0xa25b bytes) 2024-04-23T23:50:59.415 INFO:teuthology.orchestra.run.smithi092.stdout:5625 punch from 0xe39d4b9 to 0xe3a0a5c, (0x35a3 bytes) 2024-04-23T23:50:59.415 INFO:teuthology.orchestra.run.smithi092.stdout:5627 read 0xc37a1bc thru 0xc384117 (0x9f5c bytes) 2024-04-23T23:50:59.415 INFO:teuthology.orchestra.run.smithi092.stdout:5629 write 0xd21e426 thru 0xd22b4e8 (0xd0c3 bytes) 2024-04-23T23:50:59.417 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.417 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.419 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.419 INFO:teuthology.orchestra.run.smithi092.stdout:5631 write 0x6b4494b thru 0x6b4cc6c (0x8322 bytes) 2024-04-23T23:50:59.422 INFO:teuthology.orchestra.run.smithi092.stdout:5632 write 0x6915bcc thru 0x6920bd5 (0xb00a bytes) 2024-04-23T23:50:59.425 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:59.439+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:59.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:50:59.437 INFO:teuthology.orchestra.run.smithi092.stdout:5634 trunc from 0xe70be88 to 0x54b8406 2024-04-23T23:50:59.452 INFO:teuthology.orchestra.run.smithi092.stdout:5635 read 0x53e5297 thru 0x53f0207 (0xaf71 bytes) 2024-04-23T23:50:59.464 INFO:teuthology.orchestra.run.smithi092.stdout:5637 write 0xa09b9e9 thru 0xa0a7d8b (0xc3a3 bytes) 2024-04-23T23:50:59.471 INFO:teuthology.orchestra.run.smithi092.stdout:5638 write 0x26f9182 thru 0x26fa342 (0x11c1 bytes) 2024-04-23T23:50:59.473 INFO:teuthology.orchestra.run.smithi092.stdout:5639 read 0xbf51d thru 0xc9050 (0x9b34 bytes) 2024-04-23T23:50:59.473 INFO:teuthology.orchestra.run.smithi092.stdout:5641 writesame 0x2a73996 thru 0x2a80113 data_size 0x5(0xc77e bytes) 2024-04-23T23:50:59.480 INFO:teuthology.orchestra.run.smithi092.stdout:5642 trunc from 0xa0a7d8c to 0xab22b87 2024-04-23T23:50:59.487 INFO:teuthology.orchestra.run.smithi092.stdout:5645 read 0xa226231 thru 0xa22f075 (0x8e45 bytes) 2024-04-23T23:50:59.487 INFO:teuthology.orchestra.run.smithi092.stdout:5646 write 0xc87187 thru 0xc894de (0x2358 bytes) 2024-04-23T23:50:59.488 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:50:59.490 INFO:teuthology.orchestra.run.smithi092.stdout:5648 trunc from 0xab22b87 to 0x8990728 2024-04-23T23:50:59.502 INFO:teuthology.orchestra.run.smithi092.stdout:5649 read 0x45a63ca thru 0x45ad21e (0x6e55 bytes) 2024-04-23T23:50:59.509 INFO:teuthology.orchestra.run.smithi092.stdout:5651 write 0x9b5d8b3 thru 0x9b6a9dc (0xd12a bytes) 2024-04-23T23:50:59.516 INFO:teuthology.orchestra.run.smithi092.stdout:5653 read 0x73930ce thru 0x7393abf (0x9f2 bytes) 2024-04-23T23:50:59.516 INFO:teuthology.orchestra.run.smithi092.stdout:5654 clone 38 order 22 su 1048576 sc 9 2024-04-23T23:50:59.526 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:50:59.540+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-04-23T23:50:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-04-23T23:50:59.543 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-04-23T23:50:59.545 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T23:50:59.545 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T23:50:59.572 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.572 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:50:59.628 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:50:59.642+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:50:59.683 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.684 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.692 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:50:59.729 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:50:59.743+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:50:59.830 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:50:59.844+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:50:59.834 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.834 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:50:59.930 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:50:59.944+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:50:59.944 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:50:59.944 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:50:59.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:00.039 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:00.053+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:00.096 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.096 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:00.147 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:00.161+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:00.234 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:00.248+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:00.248 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.249 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:00.335 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:00.349+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:00.352 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone37 from 0x9b6a9dd (overlap 0x88b24c) to 0x1b56dc 2024-04-23T23:51:00.368 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.368 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:00.436 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:00.450+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:00.473 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-04-23T23:51:00.490 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.490 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.491 INFO:teuthology.orchestra.run.smithi092.stdout:5656 read 0x59f04c0 thru 0x59f53cb (0x4f0c bytes) 2024-04-23T23:51:00.493 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.493 INFO:teuthology.orchestra.run.smithi092.stdout:5658 trunc from 0x9b6a9dd to 0x9c8d4b7 2024-04-23T23:51:00.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:00.502 INFO:teuthology.orchestra.run.smithi092.stdout:5659 read 0x860293f thru 0x8611889 (0xef4b bytes) 2024-04-23T23:51:00.502 INFO:teuthology.orchestra.run.smithi092.stdout:5661 read 0x756ac31 thru 0x757103c (0x640c bytes) 2024-04-23T23:51:00.502 INFO:teuthology.orchestra.run.smithi092.stdout:5663 write 0x137627 thru 0x1469cb (0xf3a5 bytes) 2024-04-23T23:51:00.512 INFO:teuthology.orchestra.run.smithi092.stdout:5664 read 0x257315a thru 0x2576114 (0x2fbb bytes) 2024-04-23T23:51:00.512 INFO:teuthology.orchestra.run.smithi092.stdout:5665 read 0x8855a89 thru 0x885e54c (0x8ac4 bytes) 2024-04-23T23:51:00.512 INFO:teuthology.orchestra.run.smithi092.stdout:5666 read 0x6437cb thru 0x6464ee (0x2d24 bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5669 writesame 0xb6f0d31 thru 0xb6f9e81 data_size 0x1(0x9151 bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5671 read 0x788701f thru 0x78953e6 (0xe3c8 bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5672 read 0x36f7748 thru 0x37065a2 (0xee5b bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5673 punch from 0x400e7c2 to 0x4015a32, (0x7270 bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5674 read 0xa48b3b8 thru 0xa494637 (0x9280 bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5675 read 0x61df894 thru 0x61ee4ff (0xec6c bytes) 2024-04-23T23:51:00.520 INFO:teuthology.orchestra.run.smithi092.stdout:5676 writesame 0x84c813 thru 0x855c5d data_size 0x1(0x944b bytes) 2024-04-23T23:51:00.536 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:00.550+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:00.542 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.543 INFO:teuthology.orchestra.run.smithi092.stdout:5678 read 0xb5aea41 thru 0xb5b8d12 (0xa2d2 bytes) 2024-04-23T23:51:00.543 INFO:teuthology.orchestra.run.smithi092.stdout:5679 read 0x7d2d763 thru 0x7d3cdc7 (0xf665 bytes) 2024-04-23T23:51:00.543 INFO:teuthology.orchestra.run.smithi092.stdout:5681 read 0xb4c6751 thru 0xb4cc230 (0x5ae0 bytes) 2024-04-23T23:51:00.551 INFO:teuthology.orchestra.run.smithi092.stdout:5683 trunc from 0xb6f9e82 to 0x402694b 2024-04-23T23:51:00.573 INFO:teuthology.orchestra.run.smithi092.stdout:5684 read 0x379061e thru 0x37992d5 (0x8cb8 bytes) 2024-04-23T23:51:00.586 INFO:teuthology.orchestra.run.smithi092.stdout:5685 write 0x9f34bec thru 0x9f3bdf1 (0x7206 bytes) 2024-04-23T23:51:00.589 INFO:teuthology.orchestra.run.smithi092.stdout:5687 writesame 0x9880623 thru 0x9880d7a data_size 0x14(0x758 bytes) 2024-04-23T23:51:00.592 INFO:teuthology.orchestra.run.smithi092.stdout:5688 read 0x51e207d thru 0x51e4907 (0x288b bytes) 2024-04-23T23:51:00.592 INFO:teuthology.orchestra.run.smithi092.stdout:5689 read 0x6d0db7c thru 0x6d1a689 (0xcb0e bytes) 2024-04-23T23:51:00.592 INFO:teuthology.orchestra.run.smithi092.stdout:5691 write 0x3df5b1b thru 0x3dfcd82 (0x7268 bytes) 2024-04-23T23:51:00.594 INFO:teuthology.orchestra.run.smithi092.stdout:5694 writesame 0x10697cf thru 0x1079179 data_size 0x5(0xf9ab bytes) 2024-04-23T23:51:00.613 INFO:teuthology.orchestra.run.smithi092.stdout:5695 write 0xab3a67b thru 0xab4880f (0xe195 bytes) 2024-04-23T23:51:00.616 INFO:teuthology.orchestra.run.smithi092.stdout:5697 write 0x3992a64 thru 0x39a1364 (0xe901 bytes) 2024-04-23T23:51:00.618 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.618 INFO:teuthology.orchestra.run.smithi092.stdout:5700 read 0xa2a68a1 thru 0xa2ab5aa (0x4d0a bytes) 2024-04-23T23:51:00.619 INFO:teuthology.orchestra.run.smithi092.stdout:5701 read 0x7ef97e4 thru 0x7effb2c (0x6349 bytes) 2024-04-23T23:51:00.619 INFO:teuthology.orchestra.run.smithi092.stdout:5702 punch from 0x8c3fdbe to 0x8c4c515, (0xc757 bytes) 2024-04-23T23:51:00.619 INFO:teuthology.orchestra.run.smithi092.stdout:5705 punch from 0x3e8454d to 0x3e93e9f, (0xf952 bytes) 2024-04-23T23:51:00.619 INFO:teuthology.orchestra.run.smithi092.stdout:5706 write 0x4edde9e thru 0x4edf4cb (0x162e bytes) 2024-04-23T23:51:00.620 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.620 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.621 INFO:teuthology.orchestra.run.smithi092.stdout:5707 read 0x98dd534 thru 0x98e0d07 (0x37d4 bytes) 2024-04-23T23:51:00.622 INFO:teuthology.orchestra.run.smithi092.stdout:5709 read 0x95e39a7 thru 0x95f2a05 (0xf05f bytes) 2024-04-23T23:51:00.622 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.622 INFO:teuthology.orchestra.run.smithi092.stdout:5715 write 0x2a6b46a thru 0x2a6f644 (0x41db bytes) 2024-04-23T23:51:00.624 INFO:teuthology.orchestra.run.smithi092.stdout:5716 writesame 0x18b684e thru 0x18c108e data_size 0x1(0xa841 bytes) 2024-04-23T23:51:00.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:00.649 INFO:teuthology.orchestra.run.smithi092.stdout:5717 writesame 0x66a4b7d thru 0x66adca4 data_size 0x14(0x9128 bytes) 2024-04-23T23:51:00.649 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:00.663+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:00.654 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.654 INFO:teuthology.orchestra.run.smithi092.stdout:5719 write 0x51489dd thru 0x5151a49 (0x906d bytes) 2024-04-23T23:51:00.665 INFO:teuthology.orchestra.run.smithi092.stdout:5720 write 0xda8ffcd thru 0xda9ba35 (0xba69 bytes) 2024-04-23T23:51:00.668 INFO:teuthology.orchestra.run.smithi092.stdout:5721 writesame 0x6f23aee thru 0x6f2c6d4 data_size 0x1d(0x8be7 bytes) 2024-04-23T23:51:00.672 INFO:teuthology.orchestra.run.smithi092.stdout:5722 write 0x3a774bf thru 0x3a7ac5d (0x379f bytes) 2024-04-23T23:51:00.674 INFO:teuthology.orchestra.run.smithi092.stdout:5724 punch from 0x3a8dd44 to 0x3a93932, (0x5bee bytes) 2024-04-23T23:51:00.674 INFO:teuthology.orchestra.run.smithi092.stdout:5725 write 0x2382205 thru 0x238f3f3 (0xd1ef bytes) 2024-04-23T23:51:00.675 INFO:teuthology.orchestra.run.smithi092.stdout:5727 writesame 0xc7e6ada thru 0xc7f692e data_size 0xb(0xfe55 bytes) 2024-04-23T23:51:00.689 INFO:teuthology.orchestra.run.smithi092.stdout:5728 trunc from 0xda9ba36 to 0x6cd714f 2024-04-23T23:51:00.719 INFO:teuthology.orchestra.run.smithi092.stdout:5729 write 0xb1d47cc thru 0xb1d5397 (0xbcc bytes) 2024-04-23T23:51:00.729 INFO:teuthology.orchestra.run.smithi092.stdout:5730 write 0xc174d31 thru 0xc179235 (0x4505 bytes) 2024-04-23T23:51:00.732 INFO:teuthology.orchestra.run.smithi092.stdout:5733 read 0x2897f3a thru 0x28a4444 (0xc50b bytes) 2024-04-23T23:51:00.733 INFO:teuthology.orchestra.run.smithi092.stdout:5734 read 0x29edf34 thru 0x29f4b95 (0x6c62 bytes) 2024-04-23T23:51:00.733 INFO:teuthology.orchestra.run.smithi092.stdout:5735 write 0x498054 thru 0x4a055f (0x850c bytes) 2024-04-23T23:51:00.737 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:00.751+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:00.742 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.742 INFO:teuthology.orchestra.run.smithi092.stdout:5737 read 0x6032999 thru 0x603bbdf (0x9247 bytes) 2024-04-23T23:51:00.746 INFO:teuthology.orchestra.run.smithi092.stdout:5738 trunc from 0xc179236 to 0x9028162 2024-04-23T23:51:00.760 INFO:teuthology.orchestra.run.smithi092.stdout:5739 write 0x677b915 thru 0x67837c2 (0x7eae bytes) 2024-04-23T23:51:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:5741 punch from 0x4aa4974 to 0x4aa9ff9, (0x5685 bytes) 2024-04-23T23:51:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:5742 punch from 0x643cfc7 to 0x6446fef, (0xa028 bytes) 2024-04-23T23:51:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:5743 read 0xe13ac9 thru 0xe22acd (0xf005 bytes) 2024-04-23T23:51:00.764 INFO:teuthology.orchestra.run.smithi092.stdout:5744 read 0x31f1ff5 thru 0x31f5f50 (0x3f5c bytes) 2024-04-23T23:51:00.764 INFO:teuthology.orchestra.run.smithi092.stdout:5745 punch from 0x56f9364 to 0x5701b8c, (0x8828 bytes) 2024-04-23T23:51:00.764 INFO:teuthology.orchestra.run.smithi092.stdout:5746 punch from 0x6439960 to 0x643a88a, (0xf2a bytes) 2024-04-23T23:51:00.764 INFO:teuthology.orchestra.run.smithi092.stdout:5747 writesame 0x4398d35 thru 0x43a7551 data_size 0x1d(0xe81d bytes) 2024-04-23T23:51:00.766 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.766 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.767 INFO:teuthology.orchestra.run.smithi092.stdout:5749 read 0x6f9644b thru 0x6f9f46f (0x9025 bytes) 2024-04-23T23:51:00.767 INFO:teuthology.orchestra.run.smithi092.stdout:5750 read 0x36d06aa thru 0x36d44a8 (0x3dff bytes) 2024-04-23T23:51:00.773 INFO:teuthology.orchestra.run.smithi092.stdout:5751 write 0x950d8ed thru 0x951c59c (0xecb0 bytes) 2024-04-23T23:51:00.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:00.779 INFO:teuthology.orchestra.run.smithi092.stdout:5752 trunc from 0x951c59d to 0xc073da6 2024-04-23T23:51:00.798 INFO:teuthology.orchestra.run.smithi092.stdout:5754 trunc from 0xc073da6 to 0xf309f1 2024-04-23T23:51:00.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:00.838 INFO:teuthology.orchestra.run.smithi092.stdout:5755 write 0x348b81c thru 0x349010e (0x48f3 bytes) 2024-04-23T23:51:00.840 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:00.854+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:00.840 INFO:teuthology.orchestra.run.smithi092.stdout:5756 read 0x255a603 thru 0x2562e44 (0x8842 bytes) 2024-04-23T23:51:00.840 INFO:teuthology.orchestra.run.smithi092.stdout:5758 write 0x25e102c thru 0x25e2785 (0x175a bytes) 2024-04-23T23:51:00.844 INFO:teuthology.orchestra.run.smithi092.stdout:5759 read 0x23f4792 thru 0x23fcc6d (0x84dc bytes) 2024-04-23T23:51:00.845 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.845 INFO:teuthology.orchestra.run.smithi092.stdout:5761 read 0xfdcd7b thru 0xfe51e9 (0x846f bytes) 2024-04-23T23:51:00.845 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.845 INFO:teuthology.orchestra.run.smithi092.stdout:5763 read 0x1546098 thru 0x15532fe (0xd267 bytes) 2024-04-23T23:51:00.845 INFO:teuthology.orchestra.run.smithi092.stdout:5764 write 0x29650eb thru 0x2972095 (0xcfab bytes) 2024-04-23T23:51:00.848 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.848 INFO:teuthology.orchestra.run.smithi092.stdout:5767 punch from 0x212140d to 0x2122271, (0xe64 bytes) 2024-04-23T23:51:00.849 INFO:teuthology.orchestra.run.smithi092.stdout:5768 read 0x2c0c798 thru 0x2c1335c (0x6bc5 bytes) 2024-04-23T23:51:00.868 INFO:teuthology.orchestra.run.smithi092.stdout:5769 writesame 0xde4301d thru 0xde51ab1 data_size 0x17(0xea95 bytes) 2024-04-23T23:51:00.868 INFO:teuthology.orchestra.run.smithi092.stdout:5770 write 0x8625944 thru 0x862e761 (0x8e1e bytes) 2024-04-23T23:51:00.871 INFO:teuthology.orchestra.run.smithi092.stdout:5771 punch from 0x1291699 to 0x129badd, (0xa444 bytes) 2024-04-23T23:51:00.871 INFO:teuthology.orchestra.run.smithi092.stdout:5772 read 0x79be0b5 thru 0x79be8a6 (0x7f2 bytes) 2024-04-23T23:51:00.871 INFO:teuthology.orchestra.run.smithi092.stdout:5773 write 0xc66d90b thru 0xc674c11 (0x7307 bytes) 2024-04-23T23:51:00.874 INFO:teuthology.orchestra.run.smithi092.stdout:5774 trunc from 0xde51ab2 to 0xe598eb3 2024-04-23T23:51:00.881 INFO:teuthology.orchestra.run.smithi092.stdout:5776 read 0x42ac72b thru 0x42ae5ca (0x1ea0 bytes) 2024-04-23T23:51:00.881 INFO:teuthology.orchestra.run.smithi092.stdout:5777 read 0x6407172 thru 0x640f1d6 (0x8065 bytes) 2024-04-23T23:51:00.881 INFO:teuthology.orchestra.run.smithi092.stdout:5781 read 0x2392616 thru 0x2397160 (0x4b4b bytes) 2024-04-23T23:51:00.881 INFO:teuthology.orchestra.run.smithi092.stdout:5784 read 0xa02e2b0 thru 0xa03223a (0x3f8b bytes) 2024-04-23T23:51:00.882 INFO:teuthology.orchestra.run.smithi092.stdout:5785 writesame 0x702e66 thru 0x712080 data_size 0x1(0xf21b bytes) 2024-04-23T23:51:00.900 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:00.900 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:00.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:00.928 INFO:teuthology.orchestra.run.smithi092.stdout:5786 punch from 0x102ab7c to 0x1032815, (0x7c99 bytes) 2024-04-23T23:51:00.928 INFO:teuthology.orchestra.run.smithi092.stdout:5787 read 0x80d0780 thru 0x80d9b01 (0x9382 bytes) 2024-04-23T23:51:00.932 INFO:teuthology.orchestra.run.smithi092.stdout:5788 trunc from 0xe598eb3 to 0xa417995 2024-04-23T23:51:00.940 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:00.954+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:00.956 INFO:teuthology.orchestra.run.smithi092.stdout:5790 write 0xd273231 thru 0xd27563d (0x240d bytes) 2024-04-23T23:51:00.959 INFO:teuthology.orchestra.run.smithi092.stdout:5791 read 0x3e396d thru 0x3ec26e (0x8902 bytes) 2024-04-23T23:51:00.960 INFO:teuthology.orchestra.run.smithi092.stdout:5792 writesame 0xca4a532 thru 0xca4b8a9 data_size 0x1c(0x1378 bytes) 2024-04-23T23:51:00.963 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:00.963 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:00.963 INFO:teuthology.orchestra.run.smithi092.stdout:5793 write 0x5fc6528 thru 0x5fd19eb (0xb4c4 bytes) 2024-04-23T23:51:00.966 INFO:teuthology.orchestra.run.smithi092.stdout:5794 read 0x1d2cf54 thru 0x1d30bbc (0x3c69 bytes) 2024-04-23T23:51:00.967 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:00.967 INFO:teuthology.orchestra.run.smithi092.stdout:5796 write 0x3e63667 thru 0x3e6cfa4 (0x993e bytes) 2024-04-23T23:51:00.969 INFO:teuthology.orchestra.run.smithi092.stdout:5798 writesame 0x35e1cee thru 0x35e30c5 data_size 0x14(0x13d8 bytes) 2024-04-23T23:51:00.971 INFO:teuthology.orchestra.run.smithi092.stdout:5799 writesame 0x36e0c5e thru 0x36eef15 data_size 0x1(0xe2b8 bytes) 2024-04-23T23:51:00.983 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:00.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:01.001 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:01.003 INFO:teuthology.orchestra.run.smithi092.stdout:5800 punch from 0xbc58ba5 to 0xbc61694, (0x8aef bytes) 2024-04-23T23:51:01.008 INFO:teuthology.orchestra.run.smithi092.stdout:5801 trunc from 0xd27563e to 0x91b1448 2024-04-23T23:51:01.022 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:01.024 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.024 INFO:teuthology.orchestra.run.smithi092.stdout:5803 punch from 0x2f647e2 to 0x2f690aa, (0x48c8 bytes) 2024-04-23T23:51:01.024 INFO:teuthology.orchestra.run.smithi092.stdout:5804 read 0x78a63b4 thru 0x78b48d0 (0xe51d bytes) 2024-04-23T23:51:01.029 INFO:teuthology.orchestra.run.smithi092.stdout:5805 write 0x9e5950b thru 0x9e5f976 (0x646c bytes) 2024-04-23T23:51:01.032 INFO:teuthology.orchestra.run.smithi092.stdout:5808 write 0x5d38d6 thru 0x5d7d57 (0x4482 bytes) 2024-04-23T23:51:01.037 INFO:teuthology.orchestra.run.smithi092.stdout:5809 trunc from 0x9e5f977 to 0x733e964 2024-04-23T23:51:01.041 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:01.055+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:01.042 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:01.048 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:01.049 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:5812 read 0x2eeb74a thru 0x2eef16e (0x3a25 bytes) 2024-04-23T23:51:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:5814 read 0x4160e15 thru 0x41617ba (0x9a6 bytes) 2024-04-23T23:51:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:5816 read 0x673ae31 thru 0x6747f8e (0xd15e bytes) 2024-04-23T23:51:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.061 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:01.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:01.065 INFO:teuthology.orchestra.run.smithi092.stdout:5819 write 0xcd26bb6 thru 0xcd30187 (0x95d2 bytes) 2024-04-23T23:51:01.070 INFO:teuthology.orchestra.run.smithi092.stdout:5820 trunc from 0xcd30188 to 0xaa51da4 2024-04-23T23:51:01.081 INFO:teuthology.orchestra.run.smithi092.stdout:5821 read 0x20ce525 thru 0x20dcbeb (0xe6c7 bytes) 2024-04-23T23:51:01.081 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:01.082 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:01.082 INFO:teuthology.orchestra.run.smithi092.stdout:5822 read 0x4b244a9 thru 0x4b2e8f7 (0xa44f bytes) 2024-04-23T23:51:01.082 INFO:teuthology.orchestra.run.smithi092.stdout:5823 punch from 0x1a6c6e to 0x1aad34, (0x40c6 bytes) 2024-04-23T23:51:01.086 INFO:teuthology.orchestra.run.smithi092.stdout:5825 trunc from 0xaa51da4 to 0x6ba8a6d 2024-04-23T23:51:01.099 INFO:teuthology.orchestra.run.smithi092.stdout:5826 writesame 0x7e91c14 thru 0x7e98953 data_size 0x2e(0x6d40 bytes) 2024-04-23T23:51:01.101 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:01.102 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:01.107 INFO:teuthology.orchestra.run.smithi092.stdout:5827 trunc from 0x7e98954 to 0x84ffe3 2024-04-23T23:51:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:5828 write 0x3b6cbf8 thru 0x3b6f577 (0x2980 bytes) 2024-04-23T23:51:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:5830 read 0x2c358d0 thru 0x2c45841 (0xff72 bytes) 2024-04-23T23:51:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:5831 read 0x13d88d7 thru 0x13de821 (0x5f4b bytes) 2024-04-23T23:51:01.134 INFO:teuthology.orchestra.run.smithi092.stdout:5832 read 0x309ee24 thru 0x30a0f63 (0x2140 bytes) 2024-04-23T23:51:01.134 INFO:teuthology.orchestra.run.smithi092.stdout:5833 punch from 0xf83618 to 0xf893de, (0x5dc6 bytes) 2024-04-23T23:51:01.134 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.142 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:01.156+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:01.151 INFO:teuthology.orchestra.run.smithi092.stdout:5835 writesame 0xdf9953c thru 0xdf9e47b data_size 0x20(0x4f40 bytes) 2024-04-23T23:51:01.163 INFO:teuthology.orchestra.run.smithi092.stdout:5836 trunc from 0xdf9e47c to 0x496c9c8 2024-04-23T23:51:01.169 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.183 INFO:teuthology.orchestra.run.smithi092.stdout:5839 write 0xb341262 thru 0xb34a8ae (0x964d bytes) 2024-04-23T23:51:01.186 INFO:teuthology.orchestra.run.smithi092.stdout:5843 read 0x47da379 thru 0x47e089a (0x6522 bytes) 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.390641312, 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.388597329, 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.386620080999997, 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.386589448000002, 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.384646598000003, 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.382561389000003, 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.380551957999998, 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.378651990000002, 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.374603462000003, 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.372558785999999, 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.370485213000002, 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.368597057000002, 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.368576279999999, 2024-04-23T23:51:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.360584785999997, 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.358649804000002, 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.358623709, 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.976906244999999, 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:51:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:51:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.976863825999999, 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.976827869000001, 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.976803511, 2024-04-23T23:51:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.287 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:01.287 INFO:teuthology.orchestra.run.smithi092.stdout:5844 trunc from 0xb34a8af to 0x4d77554 2024-04-23T23:51:01.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:01.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:51:01.290 INFO:teuthology.orchestra.run.smithi092.stdout:5845 read 0x20bf281 thru 0x20c281e (0x359e bytes) 2024-04-23T23:51:01.290 INFO:teuthology.orchestra.run.smithi092.stdout:5847 flatten 2024-04-23T23:51:01.290 INFO:teuthology.orchestra.run.smithi092.stdout:5848 read 0x1755283 thru 0x175db28 (0x88a6 bytes) 2024-04-23T23:51:01.290 INFO:teuthology.orchestra.run.smithi092.stdout:5849 trunc from 0x4d77554 to 0x2518bc6 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5850 writesame 0x259aba2 thru 0x259e8a2 data_size 0x17(0x3d01 bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5852 trunc from 0x259e8a3 to 0xbe1e98f 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5854 punch from 0xacc264b to 0xaccd581, (0xaf36 bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5860 read 0x31e7599 thru 0x31e9698 (0x2100 bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5861 read 0x9f389c thru 0x9fd1ff (0x9964 bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5862 writesame 0xba4e5ed thru 0xba5cd97 data_size 0x3(0xe7ab bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5863 write 0x73be5f5 thru 0x73ccbcc (0xe5d8 bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5864 trunc from 0xbe1e98f to 0x2037fd7 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5865 read 0xdea03a thru 0xdee470 (0x4437 bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5866 punch from 0x1b1114e to 0x1b18bd9, (0x7a8b bytes) 2024-04-23T23:51:01.291 INFO:teuthology.orchestra.run.smithi092.stdout:5868 read 0x1324abc thru 0x1326989 (0x1ece bytes) 2024-04-23T23:51:01.299 INFO:teuthology.orchestra.run.smithi092.stdout:5869 writesame 0x71b0983 thru 0x71b2039 data_size 0x1(0x16b7 bytes) 2024-04-23T23:51:01.299 INFO:teuthology.orchestra.run.smithi092.stdout:5870 writesame 0x63618ed thru 0x636e862 data_size 0xa(0xcf76 bytes) 2024-04-23T23:51:01.299 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:01.313+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:01.318 INFO:teuthology.orchestra.run.smithi092.stdout:5871 write 0xdeebbc0 thru 0xdef2f61 (0x73a2 bytes) 2024-04-23T23:51:01.321 INFO:teuthology.orchestra.run.smithi092.stdout:5872 write 0xd43fa70 thru 0xd44b969 (0xbefa bytes) 2024-04-23T23:51:01.325 INFO:teuthology.orchestra.run.smithi092.stdout:5874 writesame 0xc78e094 thru 0xc796a3c data_size 0x11(0x89a9 bytes) 2024-04-23T23:51:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:5875 punch from 0x90d1ec2 to 0x90e06ea, (0xe828 bytes) 2024-04-23T23:51:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:5877 writesame 0x28c0e0a thru 0x28c90d4 data_size 0x1(0x82cb bytes) 2024-04-23T23:51:01.348 INFO:teuthology.orchestra.run.smithi092.stdout:5879 write 0x3abc29c thru 0x3ac0574 (0x42d9 bytes) 2024-04-23T23:51:01.352 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.352 INFO:teuthology.orchestra.run.smithi092.stdout:5881 read 0x3b98c28 thru 0x3b9d152 (0x452b bytes) 2024-04-23T23:51:01.352 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.352 INFO:teuthology.orchestra.run.smithi092.stdout:5884 read 0x81e7dc thru 0x829621 (0xae46 bytes) 2024-04-23T23:51:01.353 INFO:teuthology.orchestra.run.smithi092.stdout:5887 read 0x20a716e thru 0x20a8616 (0x14a9 bytes) 2024-04-23T23:51:01.353 INFO:teuthology.orchestra.run.smithi092.stdout:5888 write 0x5d96f74 thru 0x5d9a4be (0x354b bytes) 2024-04-23T23:51:01.356 INFO:teuthology.orchestra.run.smithi092.stdout:5891 writesame 0x2934e00 thru 0x2935154 data_size 0x1(0x355 bytes) 2024-04-23T23:51:01.359 INFO:teuthology.orchestra.run.smithi092.stdout:5892 writesame 0x6055140 thru 0x6060141 data_size 0xd(0xb002 bytes) 2024-04-23T23:51:01.373 INFO:teuthology.orchestra.run.smithi092.stdout:5896 trunc from 0xdef2f62 to 0x735ec04 2024-04-23T23:51:01.384 INFO:teuthology.orchestra.run.smithi092.stdout:5898 read 0x4bfe995 thru 0x4c061da (0x7846 bytes) 2024-04-23T23:51:01.384 INFO:teuthology.orchestra.run.smithi092.stdout:5899 punch from 0x8da1ff to 0x8e3a9f, (0x98a0 bytes) 2024-04-23T23:51:01.385 INFO:teuthology.orchestra.run.smithi092.stdout:5900 read 0x3a77b32 thru 0x3a82b13 (0xafe2 bytes) 2024-04-23T23:51:01.391 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:01.405+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:01.398 INFO:teuthology.orchestra.run.smithi092.stdout:5902 writesame 0xd437e37 thru 0xd43ac36 data_size 0x40(0x2e00 bytes) 2024-04-23T23:51:01.409 INFO:teuthology.orchestra.run.smithi092.stdout:5904 trunc from 0xd43ac37 to 0x5540a0a 2024-04-23T23:51:01.415 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:01.415 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:114550 3.4 3:211fbaaa:::rbd_header.1179d77c3b00:head [call rbd.parent_get in=13b out=30b,call rbd.parent_overlap_get in=29b out=1b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:01.429771+0000", 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00025232100000000001, 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964300000000002, 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 114550 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429771+0000", 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429771+0000", 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429772+0000", 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.094e-06 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429778+0000", 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6250000000000004e-06 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429790+0000", 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1763000000000001e-05 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429797+0000", 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2180000000000002e-06 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429833+0000", 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6387999999999998e-05 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.429881+0000", 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7500999999999998e-05 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:01.430041+0000", 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160054 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-04-23T23:51:01.420 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:01.421 INFO:teuthology.orchestra.run.smithi092.stdout:5905 trunc from 0x5540a0a to 0x7e8f942 2024-04-23T23:51:01.425 INFO:teuthology.orchestra.run.smithi092.stdout:5906 read 0x4c465e thru 0x4cb642 (0x6fe5 bytes) 2024-04-23T23:51:01.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:01.428 INFO:teuthology.orchestra.run.smithi092.stdout:5908 trunc from 0x7e8f942 to 0x8b924eb 2024-04-23T23:51:01.443 INFO:teuthology.orchestra.run.smithi092.stdout:5909 trunc from 0x8b924eb to 0x968299 2024-04-23T23:51:01.465 INFO:teuthology.orchestra.run.smithi092.stdout:5910 trunc from 0x968299 to 0x505a193 2024-04-23T23:51:01.470 INFO:teuthology.orchestra.run.smithi092.stdout:5911 write 0x142c2f3 thru 0x143303b (0x6d49 bytes) 2024-04-23T23:51:01.472 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.481 INFO:teuthology.orchestra.run.smithi092.stdout:5914 trunc from 0x505a193 to 0xcb36472 2024-04-23T23:51:01.492 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:01.506+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:01.496 INFO:teuthology.orchestra.run.smithi092.stdout:5915 trunc from 0xcb36472 to 0x4e4af0e 2024-04-23T23:51:01.501 INFO:teuthology.orchestra.run.smithi092.stdout:5916 read 0x4583e07 thru 0x458c30a (0x8504 bytes) 2024-04-23T23:51:01.502 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.502 INFO:teuthology.orchestra.run.smithi092.stdout:5919 read 0x31180e6 thru 0x311dcae (0x5bc9 bytes) 2024-04-23T23:51:01.508 INFO:teuthology.orchestra.run.smithi092.stdout:5920 trunc from 0x4e4af0e to 0xa9d6c72 2024-04-23T23:51:01.514 INFO:teuthology.orchestra.run.smithi092.stdout:5921 write 0x7dd1374 thru 0x7ddf8f0 (0xe57d bytes) 2024-04-23T23:51:01.517 INFO:teuthology.orchestra.run.smithi092.stdout:5922 read 0x2f6dfea thru 0x2f6fb93 (0x1baa bytes) 2024-04-23T23:51:01.517 INFO:teuthology.orchestra.run.smithi092.stdout:5925 write 0x5432a78 thru 0x543cefd (0xa486 bytes) 2024-04-23T23:51:01.520 INFO:teuthology.orchestra.run.smithi092.stdout:5927 write 0x8b364cb thru 0x8b3a572 (0x40a8 bytes) 2024-04-23T23:51:01.523 INFO:teuthology.orchestra.run.smithi092.stdout:5928 read 0x9c60d76 thru 0x9c6e26f (0xd4fa bytes) 2024-04-23T23:51:01.523 INFO:teuthology.orchestra.run.smithi092.stdout:5929 write 0x9c96502 thru 0x9c97e00 (0x18ff bytes) 2024-04-23T23:51:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:5931 read 0x250648e thru 0x251636b (0xfede bytes) 2024-04-23T23:51:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:5933 read 0x66b55c3 thru 0x66c134e (0xbd8c bytes) 2024-04-23T23:51:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:5934 write 0x9d9d41d thru 0x9da90a2 (0xbc86 bytes) 2024-04-23T23:51:01.530 INFO:teuthology.orchestra.run.smithi092.stdout:5935 read 0x5cd1a9e thru 0x5cdd5b8 (0xbb1b bytes) 2024-04-23T23:51:01.530 INFO:teuthology.orchestra.run.smithi092.stdout:5936 punch from 0x9197305 to 0x919af81, (0x3c7c bytes) 2024-04-23T23:51:01.530 INFO:teuthology.orchestra.run.smithi092.stdout:5937 trunc from 0xa9d6c72 to 0xa79db09 2024-04-23T23:51:01.536 INFO:teuthology.orchestra.run.smithi092.stdout:5938 punch from 0x3093c19 to 0x3096315, (0x26fc bytes) 2024-04-23T23:51:01.536 INFO:teuthology.orchestra.run.smithi092.stdout:5940 punch from 0x60e2414 to 0x60e818d, (0x5d79 bytes) 2024-04-23T23:51:01.536 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:5943 trunc from 0xa79db09 to 0x4e81cb6 2024-04-23T23:51:01.562 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:01.563 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:01.564 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:01.564 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:01.565 INFO:teuthology.orchestra.run.smithi092.stdout:5944 write 0xa4e2785 thru 0xa4ef9cd (0xd249 bytes) 2024-04-23T23:51:01.569 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xee02816 2024-04-23T23:51:01.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:01.574 INFO:teuthology.orchestra.run.smithi092.stdout:5946 trunc from 0xa4ef9ce to 0xee02816 2024-04-23T23:51:01.580 INFO:teuthology.orchestra.run.smithi092.stdout:5948 write 0x7abe0b0 thru 0x7abe477 (0x3c8 bytes) 2024-04-23T23:51:01.583 INFO:teuthology.orchestra.run.smithi092.stdout:5949 write 0x9ca27dc thru 0x9ca2a93 (0x2b8 bytes) 2024-04-23T23:51:01.586 INFO:teuthology.orchestra.run.smithi092.stdout:5951 punch from 0xa1cdd58 to 0xa1d5ef2, (0x819a bytes) 2024-04-23T23:51:01.586 INFO:teuthology.orchestra.run.smithi092.stdout:5954 read 0xa10984f thru 0xa10e00e (0x47c0 bytes) 2024-04-23T23:51:01.586 INFO:teuthology.orchestra.run.smithi092.stdout:5956 punch from 0xb8689d4 to 0xb87632b, (0xd957 bytes) 2024-04-23T23:51:01.586 INFO:teuthology.orchestra.run.smithi092.stdout:5958 writesame 0x9ad4e3e thru 0x9ad6990 data_size 0x1(0x1b53 bytes) 2024-04-23T23:51:01.592 INFO:teuthology.orchestra.run.smithi092.stdout:5960 write 0xa95358a thru 0xa962849 (0xf2c0 bytes) 2024-04-23T23:51:01.593 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:01.606+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:01.596 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.599 INFO:teuthology.orchestra.run.smithi092.stdout:5962 trunc from 0xee02816 to 0xc135b70 2024-04-23T23:51:01.612 INFO:teuthology.orchestra.run.smithi092.stdout:5963 trunc from 0xc135b70 to 0x636d90b 2024-04-23T23:51:01.632 INFO:teuthology.orchestra.run.smithi092.stdout:5966 trunc from 0x636d90b to 0x9f083cf 2024-04-23T23:51:01.638 INFO:teuthology.orchestra.run.smithi092.stdout:5967 writesame 0x2b242ff thru 0x2b2a2d5 data_size 0x7(0x5fd7 bytes) 2024-04-23T23:51:01.648 INFO:teuthology.orchestra.run.smithi092.stdout:5968 trunc from 0x9f083cf to 0x59c16b4 2024-04-23T23:51:01.661 INFO:teuthology.orchestra.run.smithi092.stdout:5969 trunc from 0x59c16b4 to 0x996fd1f 2024-04-23T23:51:01.666 INFO:teuthology.orchestra.run.smithi092.stdout:5970 writesame 0x94503bd thru 0x945e6c3 data_size 0x1(0xe307 bytes) 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:01.694 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.194016308999998, 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:01.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.193550322, 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.699 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.703 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.193043772999999, 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:01.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:01.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.192747865000001, 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:01.708 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:01.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.192560852, 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.192359181999997, 2024-04-23T23:51:01.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:01.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:01.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:01.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.192218920000002, 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:01.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:01.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.191973185000002, 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:01.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.191695660000001, 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:01.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.191453088000003, 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:01.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:01.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.191274724000003, 2024-04-23T23:51:01.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:01.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.190992133000002, 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:01.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.190799648999999, 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:01.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.190371708000001, 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.190219374999998, 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:01.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:01.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:01.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:01.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:01.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:01.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.190116291999999, 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.189799346000001, 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:01.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:01.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:01.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.189557919000002, 2024-04-23T23:51:01.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:01.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:01.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:01.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.189333372, 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.189078475000002, 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:01.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:01.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:01.787 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:01.787 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:01.707+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:01.788 INFO:teuthology.orchestra.run.smithi092.stdout:5971 read 0x13f56bd thru 0x14011dd (0xbb21 bytes) 2024-04-23T23:51:01.788 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.788 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.788 INFO:teuthology.orchestra.run.smithi092.stdout:5975 read 0xfdfc70 thru 0xfe17b8 (0x1b49 bytes) 2024-04-23T23:51:01.788 INFO:teuthology.orchestra.run.smithi092.stdout:5976 writesame 0x8e8c277 thru 0x8e972c3 data_size 0xb(0xb04d bytes) 2024-04-23T23:51:01.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:5978 read 0x1ea9699 thru 0x1eb6566 (0xcece bytes) 2024-04-23T23:51:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:5979 trunc from 0x996fd1f to 0x8d9d0e4 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5980 read 0x567aa4 thru 0x573db7 (0xc314 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5981 write 0xda1b58b thru 0xda1ebd0 (0x3646 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5982 writesame 0x35c42fb thru 0x35ce63d data_size 0x5(0xa343 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5987 write 0xd7640c5 thru 0xd7652a1 (0x11dd bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5989 punch from 0x809a387 to 0x80a7921, (0xd59a bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5990 write 0x889a920 thru 0x88a7746 (0xce27 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5991 read 0x8b7f788 thru 0x8b8f0de (0xf957 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5992 writesame 0xce3fb8c thru 0xce4b82b data_size 0x30(0xbca0 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5993 punch from 0x3d2e3cf to 0x3d3745b, (0x908c bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5995 read 0x32ab2b9 thru 0x32aeef5 (0x3c3d bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5996 writesame 0xc6faf75 thru 0xc6ffe03 data_size 0x5b(0x4e8f bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5997 writesame 0xc22100 thru 0xc2521f data_size 0x20(0x3120 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5998 write 0x10c8be1 thru 0x10d77a2 (0xebc2 bytes) 2024-04-23T23:51:01.792 INFO:teuthology.orchestra.run.smithi092.stdout:5999 writesame 0x72e9bc3 thru 0x72f6fba data_size 0x98(0xd3f8 bytes) 2024-04-23T23:51:01.793 INFO:teuthology.orchestra.run.smithi092.stdout:6000 read 0x380bbe3 thru 0x381a9a2 (0xedc0 bytes) 2024-04-23T23:51:01.795 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:01.809+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:01.803 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-04-23T23:51:01.896 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:01.909+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:01.917 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:01.917 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:01.917 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:01.918 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:01.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:01.997 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:02.011+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:02.070 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:02.070 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:02.070 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:02.070 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:02.071 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:02.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:02.098 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:02.112+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.736395676999997, 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:02.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.988736717000002, 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.188 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:51:02.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:51:02.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:51:02.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:51:02.193 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.146867411999999, 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:58072 3.9 e54/22 3:967d967a:::rbd_data.117966a158ce.0000000000000002:head v 54'280, mlcod=53'276)", 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.002510549999997, 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003273686, 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193448+0000", 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2024-04-23T23:51:02.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193458+0000", 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0628000000000001e-05 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193469+0000", 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380000000000001e-05 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193479+0000", 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-04-23T23:51:02.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193529+0000", 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-05 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193551+0000", 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196689+0000", 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031378420000000001 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196721+0000", 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2400000000000001e-05 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:02.203 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.926889182, 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:02.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:02.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:02.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.877094294999999, 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:02.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.875963543000001, 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:51:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:51:02.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:51:02.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.830469227000002, 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.827698853000001, 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:02.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63748 3.4 e58/22 3:21ed4642:::rbd_data.11798ae5b3f1.0000000000000064:head v 58'132, mlcod=58'128)", 2024-04-23T23:51:02.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.799307832, 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003193577, 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396651+0000", 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396657+0000", 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8980000000000001e-06 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396667+0000", 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0288e-05 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396673+0000", 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396718+0000", 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:51:02.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396736+0000", 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8071e-05 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399806+0000", 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003069881 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399844+0000", 2024-04-23T23:51:02.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8056999999999998e-05 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.732313024, 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:02.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:02.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:02.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:02.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.729894942000001, 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:02.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64292 3.9 e58/22 3:957b2643:::rbd_data.11798ae5b3f1.000000000000011d:head v 58'325, mlcod=58'323)", 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.666417423999999, 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003216596, 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529541+0000", 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529546+0000", 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6619999999999996e-06 2024-04-23T23:51:02.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529557+0000", 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1035e-05 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529564+0000", 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0219999999999999e-06 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529615+0000", 2024-04-23T23:51:02.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0899000000000002e-05 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529636+0000", 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1739999999999999e-05 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532716+0000", 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003079755 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532757+0000", 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64347 3.4 e58/22 3:23ee6249:::rbd_data.11798ae5b3f1.000000000000013a:head v 58'151, mlcod=58'149)", 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:51:02.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.647604830999999, 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036492059999999999, 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548354+0000", 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6899999999999996e-07 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548359+0000", 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548372+0000", 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3050999999999999e-05 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548379+0000", 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548448+0000", 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9326999999999995e-05 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548466+0000", 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659000000000002e-05 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551946+0000", 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034799850000000001 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552002+0000", 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6508999999999997e-05 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64341 3.0 e58/22 3:0635a7b0:::rbd_data.11798ae5b3f1.0000000000000138:head v 58'169, mlcod=58'166)", 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.647196928, 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032676770000000001, 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:02.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548766+0000", 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548777+0000", 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1453e-05 2024-04-23T23:51:02.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548784+0000", 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6460000000000004e-06 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548833+0000", 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8498000000000002e-05 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548852+0000", 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9142000000000001e-05 2024-04-23T23:51:02.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551992+0000", 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031404649999999998 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552029+0000", 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6381999999999998e-05 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.63960256, 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:51:02.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:51:02.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:51:02.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:67439 3.0 e58/22 3:0718c4b2:::rbd_data.117911508d31.0000000000000006:head v 58'177, mlcod=58'175)", 2024-04-23T23:51:02.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:51:02.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286833996999999, 2024-04-23T23:51:02.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036877949999999998, 2024-04-23T23:51:02.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909125+0000", 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909535+0000", 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040998899999999998 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909550+0000", 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5092000000000001e-05 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909558+0000", 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6790000000000005e-06 2024-04-23T23:51:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909610+0000", 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1996000000000002e-05 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909638+0000", 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8231000000000001e-05 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912774+0000", 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031359740000000001 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912812+0000", 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7710999999999999e-05 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.265 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:98054 3.0 e74/22 3:00ee5193:::rbd_data.1179fd9d5edc.0000000000000002:head v 74'235, mlcod=73'233)", 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6293191119999992, 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003737537, 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1100000000000002e-07 2024-04-23T23:51:02.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567428+0000", 2024-04-23T23:51:02.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078870700000000004 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567444+0000", 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.626e-05 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567456+0000", 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1902000000000001e-05 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567526+0000", 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0288000000000002e-05 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567552+0000", 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.622e-05 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.570291+0000", 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002738603 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.570376+0000", 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5246000000000003e-05 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.0111476719999999, 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:02.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.8355642210000003, 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:02.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:02.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:02.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:02.282 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:02.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:02.285 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:51:02.285 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:02.297+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:02.384 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:02.398+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:02.394 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:02.394 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:02.394 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:02.394 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:02.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:02.487 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:02.501+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:02.545 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:02.546 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:02.554 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:02.587 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:02.600+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:02.688 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:02.702+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.280586733, 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:02.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.280576852999999, 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.768 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.280366506, 2024-04-23T23:51:02.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.278535167000001, 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:02.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:02.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:02.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.276465536000003, 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:02.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:02.785 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764375299999998, 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:02.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764349643999999, 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764282154, 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.796 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:02.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:02.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764237092999998, 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.806 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.811 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764200592999998, 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.812 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:02.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:02.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:02.816 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764183633999998, 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:02.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:02.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764104949999997, 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.821 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.825 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.764037917000003, 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:02.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.763904316000001, 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:02.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.763485635999999, 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.839 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.763334993999997, 2024-04-23T23:51:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.840 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:02.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.845 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.763209611999997, 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.849 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.762994095000003, 2024-04-23T23:51:02.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.762756760999999, 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.762516865999999, 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:02.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:02.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:02.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:02.868 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:02.882+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:02.969 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:02.983+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:02.970 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:02.970 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:02.970 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:02.970 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:02.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:03.070 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:03.084+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:03.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:03.120 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:03.120 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:03.121 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:03.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:03.171 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:03.185+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.975127145000002, 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:03.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:03.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.236 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.975106265000001, 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.969248479000001, 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.241 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:03.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.969234231999998, 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:03.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.967200298999998, 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.254 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.967188356999998, 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:03.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:03.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:03.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.965211222999997, 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:03.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:03.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.965183138999997, 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:03.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.965170604000001, 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:03.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:03.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.963229515999998, 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.963218169999998, 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.963207316000002, 2024-04-23T23:51:03.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:03.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:03.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.963196025000002, 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:03.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:03.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.961157252, 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:03.297 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.961141269000002, 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.961120649999998, 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:03.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:03.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.961100651999999, 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:03.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.961082372, 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:03.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:03.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.960080173999998, 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.960068675000002, 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:03.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:03.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:03.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:03.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:03.323 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:03.324 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:03.325 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:03.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:03.327 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:03.327 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:03.328 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:03.328 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:03.338+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:03.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:03.330 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:03.349 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:03.349 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:03.367 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:03.367 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:03.385 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:03.385 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:03.426 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:03.440+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:03.436 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:03.436 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:03.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:03.528 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:03.542+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:03.586 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:03.586 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:03.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:03.627 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:03.641+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:03.729 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:03.743+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:03.737 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:03.737 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:03.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:03.830 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:03.844+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:03.846 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:03.846 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:03.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:03.931 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:03.944+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:03.957 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:03.958 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:03.966 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:04.032 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:04.045+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:04.068 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.068 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:04.133 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:04.147+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:04.188 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.189 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:04.234 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:04.248+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:04.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:51:04.335 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:04.349+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:04.354 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.354 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:04.436 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:04.450+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:04.476 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.476 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:04.537 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:04.551+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:04.546 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:04.546 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:04.546 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-23T23:51:04.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-23T23:51:04.590 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.590 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:04.638 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:04.653+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:04.739 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:04.753+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:04.743 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.744 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:04.840 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:04.854+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:51:04.841 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 84, 2024-04-23T23:51:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "6a51c05a-822c-4999-80c7-e8408cd2a202", 2024-04-23T23:51:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-04-23T23:49:23.155452+0000", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-04-23T23:51:04.289860+0000", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-04-23T23:50:32.112122+0000", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-04-23T23:49:26.183177+0000", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5801984, 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "noscrub", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 6, 2024-04-23T23:51:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 3, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "squid", 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_crimson": false, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-04-23T23:51:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:35.387984+0000", 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 1, 2024-04-23T23:51:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 1, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 1, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 1, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 1, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "18", 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:51:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:51:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:51:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-04-23T23:51:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 8, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 8, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.25, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 2, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 2, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-04-23T23:51:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "rbd", 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:41.909447+0000", 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:51:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 8, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 8, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 8, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 8, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 8, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "21", 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:51:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:51:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:51:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.9900000095367432, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.9900000095367432, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.87999999523162842, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.75, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.75, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "pool_client.0", 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:49.482932+0000", 2024-04-23T23:51:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-04-23T23:51:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "84", 2024-04-23T23:51:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 40, 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 84, 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:51:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:51:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-04-23T23:51:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.5, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.5, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 1, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.5, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.5, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-04-23T23:51:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "6a84eb3e-7d8a-4436-956d-f7da68d06a21", 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 58, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-04-23T23:51:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6820/3364976130", 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6824/3364976130", 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3364976130", 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6828/3364976130", 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "be35c480-9a84-4846-9f0b-043c7c2c9f4a", 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 59, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6817/2924722109", 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/2924722109", 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6829/2924722109", 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6825/2924722109", 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "64e8bbbb-690a-4267-a030-30ec66f30bfa", 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 56, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-04-23T23:51:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-04-23T23:51:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-04-23T23:51:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6801/3952319845", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6803/3952319845", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3952319845", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3952319845", 2024-04-23T23:51:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "692beda9-1302-4d96-81ab-62c40a12e3f1", 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 15, 2024-04-23T23:51:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 38, 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 56, 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 59, 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 39, 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-04-23T23:51:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6809/1499077312", 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6811/1499077312", 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1499077312", 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1499077312", 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "705f6b1f-c35d-4ed8-a2c9-70b9324691e3", 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 22, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6824", 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6825", 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6826", 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6827", 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6830", 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6831", 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6828", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6829", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6825/1281396523", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6827/1281396523", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1281396523", 2024-04-23T23:51:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1281396523", 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "eb37446d-b5c9-4200-a1e9-38c1f6268401", 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 65, 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6800", 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6801", 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6802", 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6803", 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6806", 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6807", 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6804", 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6805", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6801/612886539", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6803/612886539", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6807/612886539", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6805/612886539", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "c38a4da5-1017-4bb9-86ab-999ea57999c9", 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 58, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6816", 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6817", 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6818", 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6819", 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6822", 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6823", 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6820", 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6821", 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6817/2243867625", 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6819/2243867625", 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6823/2243867625", 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6821/2243867625", 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-04-23T23:51:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "a921d465-920a-4ad0-839f-d97cd5e6ce62", 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 58, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6808", 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6809", 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6810", 2024-04-23T23:51:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6811", 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6814", 2024-04-23T23:51:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6815", 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6812", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6813", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6809/3209176445", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6811/3209176445", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3209176445", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3209176445", 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.405502+0000", 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.334094+0000", 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.268468+0000", 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.367807+0000", 2024-04-23T23:51:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 39 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.328752+0000", 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.255936+0000", 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.298943+0000", 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.352632+0000", 2024-04-23T23:51:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [ 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "3.1c", 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 4, 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 1 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_primaries": [], 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": {}, 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [ 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "snaps": [ 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "begin": 38, 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "length": 3 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [ 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:51:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "snaps": [ 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "begin": 38, 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "length": 1 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-04-23T23:51:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-04-23T23:51:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-04-23T23:51:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-04-23T23:51:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:04.918 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:04.920 INFO:tasks.thrashosds.thrasher:j is {'epoch': 84, 'fsid': '6a51c05a-822c-4999-80c7-e8408cd2a202', 'created': '2024-04-23T23:49:23.155452+0000', 'modified': '2024-04-23T23:51:04.289860+0000', 'last_up_change': '2024-04-23T23:50:32.112122+0000', 'last_in_change': '2024-04-23T23:49:26.183177+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T23:49:35.387984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-23T23:49:41.909447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-04-23T23:49:49.482932+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6a84eb3e-7d8a-4436-956d-f7da68d06a21', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6820', 'nonce': 3364976130}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6824', 'nonce': 3364976130}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3364976130}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6828', 'nonce': 3364976130}]}, 'public_addr': '172.21.15.92:6820/3364976130', 'cluster_addr': '172.21.15.92:6824/3364976130', 'heartbeat_back_addr': '172.21.15.92:6831/3364976130', 'heartbeat_front_addr': '172.21.15.92:6828/3364976130', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'be35c480-9a84-4846-9f0b-043c7c2c9f4a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2924722109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6819', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 2924722109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6827', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 2924722109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6823', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 2924722109}]}, 'public_addr': '172.21.15.92:6817/2924722109', 'cluster_addr': '172.21.15.92:6821/2924722109', 'heartbeat_back_addr': '172.21.15.92:6829/2924722109', 'heartbeat_front_addr': '172.21.15.92:6825/2924722109', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64e8bbbb-690a-4267-a030-30ec66f30bfa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3952319845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3952319845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3952319845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3952319845}]}, 'public_addr': '172.21.15.92:6801/3952319845', 'cluster_addr': '172.21.15.92:6803/3952319845', 'heartbeat_back_addr': '172.21.15.92:6807/3952319845', 'heartbeat_front_addr': '172.21.15.92:6805/3952319845', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '692beda9-1302-4d96-81ab-62c40a12e3f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 38, 'up_from': 56, 'up_thru': 59, 'down_at': 39, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1499077312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1499077312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1499077312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1499077312}]}, 'public_addr': '172.21.15.92:6809/1499077312', 'cluster_addr': '172.21.15.92:6811/1499077312', 'heartbeat_back_addr': '172.21.15.92:6815/1499077312', 'heartbeat_front_addr': '172.21.15.92:6813/1499077312', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '705f6b1f-c35d-4ed8-a2c9-70b9324691e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1281396523}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1281396523}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1281396523}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1281396523}]}, 'public_addr': '172.21.15.119:6825/1281396523', 'cluster_addr': '172.21.15.119:6827/1281396523', 'heartbeat_back_addr': '172.21.15.119:6831/1281396523', 'heartbeat_front_addr': '172.21.15.119:6829/1281396523', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb37446d-b5c9-4200-a1e9-38c1f6268401', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 612886539}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 612886539}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 612886539}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 612886539}]}, 'public_addr': '172.21.15.119:6801/612886539', 'cluster_addr': '172.21.15.119:6803/612886539', 'heartbeat_back_addr': '172.21.15.119:6807/612886539', 'heartbeat_front_addr': '172.21.15.119:6805/612886539', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c38a4da5-1017-4bb9-86ab-999ea57999c9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 2243867625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 2243867625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 2243867625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 2243867625}]}, 'public_addr': '172.21.15.119:6817/2243867625', 'cluster_addr': '172.21.15.119:6819/2243867625', 'heartbeat_back_addr': '172.21.15.119:6823/2243867625', 'heartbeat_front_addr': '172.21.15.119:6821/2243867625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a921d465-920a-4ad0-839f-d97cd5e6ce62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3209176445}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3209176445}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3209176445}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3209176445}]}, 'public_addr': '172.21.15.119:6809/3209176445', 'cluster_addr': '172.21.15.119:6811/3209176445', 'heartbeat_back_addr': '172.21.15.119:6815/3209176445', 'heartbeat_front_addr': '172.21.15.119:6813/3209176445', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.405502+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.334094+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.268468+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.367807+0000', 'dead_epoch': 39}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.328752+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.255936+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.298943+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.352632+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 38, 'length': 3}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 38, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-23T23:51:04.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:51:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:04.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:04.963 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:04.977+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:05.041 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:05.056+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:05.068 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:05.068 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:05.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:05.142 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:05.156+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:05.184 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:05.184 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:05.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:05.243 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:05.257+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:05.256 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:51:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:51:05.268 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-04-23T23:51:04.298386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1402829307,"num_objects":1796,"num_object_clones":433,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":722,"num_read":143163,"num_read_kb":589177,"num_write":15147,"num_write_kb":295466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":15046498,"num_keys_recovered":126,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13143,"ondisk_log_size":13143,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":633120,"kb_used_data":334332,"kb_used_omap":21,"kb_used_meta":298026,"kb_avail":754341600,"statfs":{"total":773094113280,"available":772445798400,"internally_reserved":0,"allocated":342355968,"data_stored":413380544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":305178997},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":116734619,"num_objects":159,"num_object_clones":53,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":58,"num_read":17234,"num_read_kb":131001,"num_write":1070,"num_write_kb":49609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.928119"},"pg_stats":[{"pgid":"3.1f","version":"81'533","reported_seq":8129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.542651+0000","last_change":"2024-04-23T23:49:53.323402+0000","last_active":"2024-04-23T23:51:00.542651+0000","last_peered":"2024-04-23T23:51:00.542651+0000","last_clean":"2024-04-23T23:51:00.542651+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:51:00.542651+0000","last_undegraded":"2024-04-23T23:51:00.542651+0000","last_fullsized":"2024-04-23T23:51:00.542651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:50.059619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085782299999999996,"stat_sum":{"num_bytes":22845430,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":23,"num_read":9596,"num_read_kb":22649,"num_write":700,"num_write_kb":10625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"81'671","reported_seq":3567,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.827084+0000","last_change":"2024-04-23T23:50:36.142638+0000","last_active":"2024-04-23T23:51:00.827084+0000","last_peered":"2024-04-23T23:51:00.827084+0000","last_clean":"2024-04-23T23:51:00.827084+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:51:00.827084+0000","last_undegraded":"2024-04-23T23:51:00.827084+0000","last_fullsized":"2024-04-23T23:51:00.827084+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:05:09.687143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067565599999999996,"stat_sum":{"num_bytes":119646292,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":5062,"num_read_kb":12059,"num_write":752,"num_write_kb":8176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"77'315","reported_seq":2915,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.273210+0000","last_change":"2024-04-23T23:49:53.328510+0000","last_active":"2024-04-23T23:50:59.273210+0000","last_peered":"2024-04-23T23:50:59.273210+0000","last_clean":"2024-04-23T23:50:59.273210+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:50:59.273210+0000","last_undegraded":"2024-04-23T23:50:59.273210+0000","last_fullsized":"2024-04-23T23:50:59.273210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:36:37.264514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018779,"stat_sum":{"num_bytes":23568518,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":3039,"num_read_kb":15742,"num_write":342,"num_write_kb":4533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"80'559","reported_seq":5511,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.387470+0000","last_change":"2024-04-23T23:50:42.176115+0000","last_active":"2024-04-23T23:50:59.387470+0000","last_peered":"2024-04-23T23:50:59.387470+0000","last_clean":"2024-04-23T23:50:59.387470+0000","last_became_active":"2024-04-23T23:50:42.175291+0000","last_became_peered":"2024-04-23T23:50:42.175291+0000","last_unstale":"2024-04-23T23:50:59.387470+0000","last_undegraded":"2024-04-23T23:50:59.387470+0000","last_fullsized":"2024-04-23T23:50:59.387470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:03.929945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099769099999999999,"stat_sum":{"num_bytes":70706992,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":6463,"num_read_kb":29044,"num_write":656,"num_write_kb":14895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2296821,"num_keys_recovered":67,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'227","reported_seq":3385,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.280916+0000","last_change":"2024-04-23T23:49:53.322810+0000","last_active":"2024-04-23T23:51:00.280916+0000","last_peered":"2024-04-23T23:51:00.280916+0000","last_clean":"2024-04-23T23:51:00.280916+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:51:00.280916+0000","last_undegraded":"2024-04-23T23:51:00.280916+0000","last_fullsized":"2024-04-23T23:51:00.280916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:48.630171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079418399999999997,"stat_sum":{"num_bytes":53755830,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":3626,"num_read_kb":16996,"num_write":205,"num_write_kb":4951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"77'230","reported_seq":3670,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.372388+0000","last_change":"2024-04-23T23:49:53.322527+0000","last_active":"2024-04-23T23:50:59.372388+0000","last_peered":"2024-04-23T23:50:59.372388+0000","last_clean":"2024-04-23T23:50:59.372388+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:50:59.372388+0000","last_undegraded":"2024-04-23T23:50:59.372388+0000","last_fullsized":"2024-04-23T23:50:59.372388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:27.425762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041081400000000001,"stat_sum":{"num_bytes":17301003,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":4043,"num_read_kb":9759,"num_write":242,"num_write_kb":2761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"81'699","reported_seq":5793,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.409429+0000","last_change":"2024-04-23T23:50:36.142889+0000","last_active":"2024-04-23T23:51:00.409429+0000","last_peered":"2024-04-23T23:51:00.409429+0000","last_clean":"2024-04-23T23:51:00.409429+0000","last_became_active":"2024-04-23T23:50:36.141286+0000","last_became_peered":"2024-04-23T23:50:36.141286+0000","last_unstale":"2024-04-23T23:51:00.409429+0000","last_undegraded":"2024-04-23T23:51:00.409429+0000","last_fullsized":"2024-04-23T23:51:00.409429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:32.504693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038598300000000001,"stat_sum":{"num_bytes":16890340,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":7810,"num_read_kb":18567,"num_write":1009,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":371151,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"81'553","reported_seq":2806,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.825331+0000","last_change":"2024-04-23T23:50:36.143465+0000","last_active":"2024-04-23T23:51:00.825331+0000","last_peered":"2024-04-23T23:51:00.825331+0000","last_clean":"2024-04-23T23:51:00.825331+0000","last_became_active":"2024-04-23T23:50:36.142717+0000","last_became_peered":"2024-04-23T23:50:36.142717+0000","last_unstale":"2024-04-23T23:51:00.825331+0000","last_undegraded":"2024-04-23T23:51:00.825331+0000","last_fullsized":"2024-04-23T23:51:00.825331+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:07.822519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064860299999999998,"stat_sum":{"num_bytes":46829389,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":15,"num_read":3890,"num_read_kb":16589,"num_write":756,"num_write_kb":10706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"81'527","reported_seq":5576,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.543272+0000","last_change":"2024-04-23T23:49:53.323222+0000","last_active":"2024-04-23T23:51:00.543272+0000","last_peered":"2024-04-23T23:51:00.543272+0000","last_clean":"2024-04-23T23:51:00.543272+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:51:00.543272+0000","last_undegraded":"2024-04-23T23:51:00.543272+0000","last_fullsized":"2024-04-23T23:51:00.543272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:18.484035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091814399999999997,"stat_sum":{"num_bytes":73388914,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":6820,"num_read_kb":33282,"num_write":649,"num_write_kb":11774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"80'343","reported_seq":5829,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.320650+0000","last_change":"2024-04-23T23:49:53.327873+0000","last_active":"2024-04-23T23:50:59.320650+0000","last_peered":"2024-04-23T23:50:59.320650+0000","last_clean":"2024-04-23T23:50:59.320650+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:50:59.320650+0000","last_undegraded":"2024-04-23T23:50:59.320650+0000","last_fullsized":"2024-04-23T23:50:59.320650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:47.294742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039462200000000002,"stat_sum":{"num_bytes":61225681,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":6852,"num_read_kb":39226,"num_write":306,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"80'274","reported_seq":1736,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.288591+0000","last_change":"2024-04-23T23:50:33.039363+0000","last_active":"2024-04-23T23:50:59.288591+0000","last_peered":"2024-04-23T23:50:59.288591+0000","last_clean":"2024-04-23T23:50:59.288591+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:50:59.288591+0000","last_undegraded":"2024-04-23T23:50:59.288591+0000","last_fullsized":"2024-04-23T23:50:59.288591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:45.385502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066676300000000001,"stat_sum":{"num_bytes":47474269,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":932,"num_read_kb":12973,"num_write":255,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"80'218","reported_seq":3972,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.270975+0000","last_change":"2024-04-23T23:49:53.322838+0000","last_active":"2024-04-23T23:50:59.270975+0000","last_peered":"2024-04-23T23:50:59.270975+0000","last_clean":"2024-04-23T23:50:59.270975+0000","last_became_active":"2024-04-23T23:49:51.320365+0000","last_became_peered":"2024-04-23T23:49:51.320365+0000","last_unstale":"2024-04-23T23:50:59.270975+0000","last_undegraded":"2024-04-23T23:50:59.270975+0000","last_fullsized":"2024-04-23T23:50:59.270975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:11:33.447546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042792500000000001,"stat_sum":{"num_bytes":43368154,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":3671,"num_read_kb":19823,"num_write":210,"num_write_kb":17227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.282082+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:50:59.282082+0000","last_peered":"2024-04-23T23:50:59.282082+0000","last_clean":"2024-04-23T23:50:59.282082+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:50:59.282082+0000","last_undegraded":"2024-04-23T23:50:59.282082+0000","last_fullsized":"2024-04-23T23:50:59.282082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"80'723","reported_seq":2034,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.280995+0000","last_change":"2024-04-23T23:49:53.322476+0000","last_active":"2024-04-23T23:51:00.280995+0000","last_peered":"2024-04-23T23:51:00.280995+0000","last_clean":"2024-04-23T23:51:00.280995+0000","last_became_active":"2024-04-23T23:49:51.317534+0000","last_became_peered":"2024-04-23T23:49:51.317534+0000","last_unstale":"2024-04-23T23:51:00.280995+0000","last_undegraded":"2024-04-23T23:51:00.280995+0000","last_fullsized":"2024-04-23T23:51:00.280995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:50:38.956306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046868700000000002,"stat_sum":{"num_bytes":31067612,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":4138,"num_read_kb":10436,"num_write":1093,"num_write_kb":4657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.279666+0000","last_change":"2024-04-23T23:49:43.259044+0000","last_active":"2024-04-23T23:51:00.279666+0000","last_peered":"2024-04-23T23:51:00.279666+0000","last_clean":"2024-04-23T23:51:00.279666+0000","last_became_active":"2024-04-23T23:49:43.258747+0000","last_became_peered":"2024-04-23T23:49:43.258747+0000","last_unstale":"2024-04-23T23:51:00.279666+0000","last_undegraded":"2024-04-23T23:51:00.279666+0000","last_fullsized":"2024-04-23T23:51:00.279666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:23.476846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"81'485","reported_seq":3913,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.974277+0000","last_change":"2024-04-23T23:50:36.144636+0000","last_active":"2024-04-23T23:51:00.974277+0000","last_peered":"2024-04-23T23:51:00.974277+0000","last_clean":"2024-04-23T23:51:00.974277+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:51:00.974277+0000","last_undegraded":"2024-04-23T23:51:00.974277+0000","last_fullsized":"2024-04-23T23:51:00.974277+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:30:29.104663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010602859999999999,"stat_sum":{"num_bytes":29412255,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":5959,"num_read_kb":15981,"num_write":549,"num_write_kb":13976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.287138+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:50:59.287138+0000","last_peered":"2024-04-23T23:50:59.287138+0000","last_clean":"2024-04-23T23:50:59.287138+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:50:59.287138+0000","last_undegraded":"2024-04-23T23:50:59.287138+0000","last_fullsized":"2024-04-23T23:50:59.287138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"81'297","reported_seq":1414,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.533444+0000","last_change":"2024-04-23T23:49:53.327533+0000","last_active":"2024-04-23T23:51:00.533444+0000","last_peered":"2024-04-23T23:51:00.533444+0000","last_clean":"2024-04-23T23:51:00.533444+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:51:00.533444+0000","last_undegraded":"2024-04-23T23:51:00.533444+0000","last_fullsized":"2024-04-23T23:51:00.533444+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:49.973004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056687800000000004,"stat_sum":{"num_bytes":76320585,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":2111,"num_read_kb":7810,"num_write":259,"num_write_kb":4090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.279649+0000","last_change":"2024-04-23T23:49:43.256177+0000","last_active":"2024-04-23T23:51:00.279649+0000","last_peered":"2024-04-23T23:51:00.279649+0000","last_clean":"2024-04-23T23:51:00.279649+0000","last_became_active":"2024-04-23T23:49:43.255854+0000","last_became_peered":"2024-04-23T23:49:43.255854+0000","last_unstale":"2024-04-23T23:51:00.279649+0000","last_undegraded":"2024-04-23T23:51:00.279649+0000","last_fullsized":"2024-04-23T23:51:00.279649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:19.944997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"80'422","reported_seq":8401,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.277908+0000","last_change":"2024-04-23T23:50:36.143902+0000","last_active":"2024-04-23T23:50:59.277908+0000","last_peered":"2024-04-23T23:50:59.277908+0000","last_clean":"2024-04-23T23:50:59.277908+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:50:59.277908+0000","last_undegraded":"2024-04-23T23:50:59.277908+0000","last_fullsized":"2024-04-23T23:50:59.277908+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:47.350135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154289,"stat_sum":{"num_bytes":49651328,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":8817,"num_read_kb":38926,"num_write":484,"num_write_kb":11249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"80'296","reported_seq":3522,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.271102+0000","last_change":"2024-04-23T23:49:53.322781+0000","last_active":"2024-04-23T23:50:59.271102+0000","last_peered":"2024-04-23T23:50:59.271102+0000","last_clean":"2024-04-23T23:50:59.271102+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:50:59.271102+0000","last_undegraded":"2024-04-23T23:50:59.271102+0000","last_fullsized":"2024-04-23T23:50:59.271102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:50:40.496658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054288899999999996,"stat_sum":{"num_bytes":20791693,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":4586,"num_read_kb":18965,"num_write":304,"num_write_kb":7655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.273149+0000","last_change":"2024-04-23T23:49:43.264520+0000","last_active":"2024-04-23T23:50:59.273149+0000","last_peered":"2024-04-23T23:50:59.273149+0000","last_clean":"2024-04-23T23:50:59.273149+0000","last_became_active":"2024-04-23T23:49:43.264218+0000","last_became_peered":"2024-04-23T23:49:43.264218+0000","last_unstale":"2024-04-23T23:50:59.273149+0000","last_undegraded":"2024-04-23T23:50:59.273149+0000","last_fullsized":"2024-04-23T23:50:59.273149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:39.497845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"81'252","reported_seq":2088,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.279515+0000","last_change":"2024-04-23T23:49:53.327404+0000","last_active":"2024-04-23T23:51:00.279515+0000","last_peered":"2024-04-23T23:51:00.279515+0000","last_clean":"2024-04-23T23:51:00.279515+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:51:00.279515+0000","last_undegraded":"2024-04-23T23:51:00.279515+0000","last_fullsized":"2024-04-23T23:51:00.279515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:15.524803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043824499999999998,"stat_sum":{"num_bytes":26311066,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":24,"num_read":2015,"num_read_kb":9958,"num_write":235,"num_write_kb":5926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T23:51:03.305747+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:51:03.305747+0000","last_peered":"2024-04-23T23:51:03.305747+0000","last_clean":"2024-04-23T23:51:03.305747+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:51:03.305747+0000","last_undegraded":"2024-04-23T23:51:03.305747+0000","last_fullsized":"2024-04-23T23:51:03.305747+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"81'276","reported_seq":2645,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.280871+0000","last_change":"2024-04-23T23:49:53.322700+0000","last_active":"2024-04-23T23:51:00.280871+0000","last_peered":"2024-04-23T23:51:00.280871+0000","last_clean":"2024-04-23T23:51:00.280871+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:51:00.280871+0000","last_undegraded":"2024-04-23T23:51:00.280871+0000","last_fullsized":"2024-04-23T23:51:00.280871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:45.254180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085923399999999995,"stat_sum":{"num_bytes":31532331,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":3143,"num_read_kb":15070,"num_write":231,"num_write_kb":2553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.286940+0000","last_change":"2024-04-23T23:49:43.264821+0000","last_active":"2024-04-23T23:50:59.286940+0000","last_peered":"2024-04-23T23:50:59.286940+0000","last_clean":"2024-04-23T23:50:59.286940+0000","last_became_active":"2024-04-23T23:49:43.264467+0000","last_became_peered":"2024-04-23T23:49:43.264467+0000","last_unstale":"2024-04-23T23:50:59.286940+0000","last_undegraded":"2024-04-23T23:50:59.286940+0000","last_fullsized":"2024-04-23T23:50:59.286940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:28:10.960350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"81'557","reported_seq":1618,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.824628+0000","last_change":"2024-04-23T23:50:33.055605+0000","last_active":"2024-04-23T23:51:00.824628+0000","last_peered":"2024-04-23T23:51:00.824628+0000","last_clean":"2024-04-23T23:51:00.824628+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:51:00.824628+0000","last_undegraded":"2024-04-23T23:51:00.824628+0000","last_fullsized":"2024-04-23T23:51:00.824628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:52:52.309997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011802519999999999,"stat_sum":{"num_bytes":27700363,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":25,"num_read":5716,"num_read_kb":22303,"num_write":760,"num_write_kb":14595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.273153+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:50:59.273153+0000","last_peered":"2024-04-23T23:50:59.273153+0000","last_clean":"2024-04-23T23:50:59.273153+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:50:59.273153+0000","last_undegraded":"2024-04-23T23:50:59.273153+0000","last_fullsized":"2024-04-23T23:50:59.273153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":286,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.286946+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:50:59.286946+0000","last_peered":"2024-04-23T23:50:59.286946+0000","last_clean":"2024-04-23T23:50:59.286946+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:50:59.286946+0000","last_undegraded":"2024-04-23T23:50:59.286946+0000","last_fullsized":"2024-04-23T23:50:59.286946+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"81'262","reported_seq":1145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.757220+0000","last_change":"2024-04-23T23:50:33.132092+0000","last_active":"2024-04-23T23:51:00.757220+0000","last_peered":"2024-04-23T23:51:00.757220+0000","last_clean":"2024-04-23T23:51:00.757220+0000","last_became_active":"2024-04-23T23:50:32.955054+0000","last_became_peered":"2024-04-23T23:50:32.955054+0000","last_unstale":"2024-04-23T23:51:00.757220+0000","last_undegraded":"2024-04-23T23:51:00.757220+0000","last_fullsized":"2024-04-23T23:51:00.757220+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:45.686754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041896000000000001,"stat_sum":{"num_bytes":20558438,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":4008,"num_read_kb":8262,"num_write":250,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":69682,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"79'408","reported_seq":2276,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.270917+0000","last_change":"2024-04-23T23:49:53.322582+0000","last_active":"2024-04-23T23:50:59.270917+0000","last_peered":"2024-04-23T23:50:59.270917+0000","last_clean":"2024-04-23T23:50:59.270917+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:50:59.270917+0000","last_undegraded":"2024-04-23T23:50:59.270917+0000","last_fullsized":"2024-04-23T23:50:59.270917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:14.827753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045422700000000003,"stat_sum":{"num_bytes":19295512,"num_objects":77,"num_object_clones":22,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":37,"num_read":3826,"num_read_kb":10112,"num_write":369,"num_write_kb":8253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"80'342","reported_seq":3007,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.346441+0000","last_change":"2024-04-23T23:49:53.494617+0000","last_active":"2024-04-23T23:50:59.346441+0000","last_peered":"2024-04-23T23:50:59.346441+0000","last_clean":"2024-04-23T23:50:59.346441+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:50:59.346441+0000","last_undegraded":"2024-04-23T23:50:59.346441+0000","last_fullsized":"2024-04-23T23:50:59.346441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:41:12.921585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039630999999999998,"stat_sum":{"num_bytes":62240122,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":27,"num_read":2676,"num_read_kb":16014,"num_write":342,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"81'360","reported_seq":2188,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:01.048275+0000","last_change":"2024-04-23T23:50:33.091564+0000","last_active":"2024-04-23T23:51:01.048275+0000","last_peered":"2024-04-23T23:51:01.048275+0000","last_clean":"2024-04-23T23:51:01.048275+0000","last_became_active":"2024-04-23T23:50:32.954861+0000","last_became_peered":"2024-04-23T23:50:32.954861+0000","last_unstale":"2024-04-23T23:51:01.048275+0000","last_undegraded":"2024-04-23T23:51:01.048275+0000","last_fullsized":"2024-04-23T23:51:01.048275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:09.339822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001287243,"stat_sum":{"num_bytes":32635170,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":19,"num_read":2764,"num_read_kb":21456,"num_write":404,"num_write_kb":8242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":3287017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"79'350","reported_seq":3576,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.273178+0000","last_change":"2024-04-23T23:50:36.143720+0000","last_active":"2024-04-23T23:50:59.273178+0000","last_peered":"2024-04-23T23:50:59.273178+0000","last_clean":"2024-04-23T23:50:59.273178+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:50:59.273178+0000","last_undegraded":"2024-04-23T23:50:59.273178+0000","last_fullsized":"2024-04-23T23:50:59.273178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:39:28.748074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012607440000000001,"stat_sum":{"num_bytes":45886730,"num_objects":71,"num_object_clones":21,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":31,"num_read":3747,"num_read_kb":26915,"num_write":334,"num_write_kb":15926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"80'257","reported_seq":2872,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.328960+0000","last_change":"2024-04-23T23:49:53.781509+0000","last_active":"2024-04-23T23:50:59.328960+0000","last_peered":"2024-04-23T23:50:59.328960+0000","last_clean":"2024-04-23T23:50:59.328960+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:50:59.328960+0000","last_undegraded":"2024-04-23T23:50:59.328960+0000","last_fullsized":"2024-04-23T23:50:59.328960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:01.884501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090598199999999995,"stat_sum":{"num_bytes":73194947,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":3159,"num_read_kb":23764,"num_write":260,"num_write_kb":14421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"80'469","reported_seq":3154,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.287192+0000","last_change":"2024-04-23T23:49:53.781245+0000","last_active":"2024-04-23T23:50:59.287192+0000","last_peered":"2024-04-23T23:50:59.287192+0000","last_clean":"2024-04-23T23:50:59.287192+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:50:59.287192+0000","last_undegraded":"2024-04-23T23:50:59.287192+0000","last_fullsized":"2024-04-23T23:50:59.287192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:18.785599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071364900000000001,"stat_sum":{"num_bytes":26923392,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":25,"num_read":4277,"num_read_kb":16202,"num_write":634,"num_write_kb":2694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"80'773","reported_seq":3549,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.287153+0000","last_change":"2024-04-23T23:49:53.781604+0000","last_active":"2024-04-23T23:50:59.287153+0000","last_peered":"2024-04-23T23:50:59.287153+0000","last_clean":"2024-04-23T23:50:59.287153+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:50:59.287153+0000","last_undegraded":"2024-04-23T23:50:59.287153+0000","last_fullsized":"2024-04-23T23:50:59.287153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:50.814061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097850600000000008,"stat_sum":{"num_bytes":71161956,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":26,"num_read":5593,"num_read_kb":22001,"num_write":1166,"num_write_kb":9477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"80'258","reported_seq":5103,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.283622+0000","last_change":"2024-04-23T23:49:53.494621+0000","last_active":"2024-04-23T23:50:59.283622+0000","last_peered":"2024-04-23T23:50:59.283622+0000","last_clean":"2024-04-23T23:50:59.283622+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:50:59.283622+0000","last_undegraded":"2024-04-23T23:50:59.283622+0000","last_fullsized":"2024-04-23T23:50:59.283622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:21:27.235787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400901,"stat_sum":{"num_bytes":25702233,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":5258,"num_read_kb":16611,"num_write":201,"num_write_kb":3602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"80'331","reported_seq":3718,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.287014+0000","last_change":"2024-04-23T23:49:53.780895+0000","last_active":"2024-04-23T23:50:59.287014+0000","last_peered":"2024-04-23T23:50:59.287014+0000","last_clean":"2024-04-23T23:50:59.287014+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:50:59.287014+0000","last_undegraded":"2024-04-23T23:50:59.287014+0000","last_fullsized":"2024-04-23T23:50:59.287014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:13.342552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049304299999999998,"stat_sum":{"num_bytes":26017447,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":18,"num_read":3904,"num_read_kb":10346,"num_write":312,"num_write_kb":4619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"77'369","reported_seq":1706,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T23:51:00.280918+0000","last_change":"2024-04-23T23:49:53.323023+0000","last_active":"2024-04-23T23:51:00.280918+0000","last_peered":"2024-04-23T23:51:00.280918+0000","last_clean":"2024-04-23T23:51:00.280918+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:51:00.280918+0000","last_undegraded":"2024-04-23T23:51:00.280918+0000","last_fullsized":"2024-04-23T23:51:00.280918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:13:18.901059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000902432,"stat_sum":{"num_bytes":66999041,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":17,"num_read":1881,"num_read_kb":16594,"num_write":370,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"77'390","reported_seq":2487,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T23:50:59.287037+0000","last_change":"2024-04-23T23:49:53.780967+0000","last_active":"2024-04-23T23:50:59.287037+0000","last_peered":"2024-04-23T23:50:59.287037+0000","last_clean":"2024-04-23T23:50:59.287037+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:50:59.287037+0000","last_undegraded":"2024-04-23T23:50:59.287037+0000","last_fullsized":"2024-04-23T23:50:59.287037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:10:35.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053415600000000004,"stat_sum":{"num_bytes":41835906,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":3665,"num_read_kb":14642,"num_write":349,"num_write_kb":15969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1402238939,"num_objects":1794,"num_object_clones":433,"num_object_copies":3588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":722,"num_read":143047,"num_read_kb":589077,"num_write":14988,"num_write_kb":293050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":15046498,"num_keys_recovered":126,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":338894848,"data_stored":414728994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":13026,"ondisk_log_size":13026,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168583,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105404,"kb_used_data":68336,"kb_used_omap":10,"kb_used_meta":36853,"kb_avail":94266436,"statfs":{"total":96636764160,"available":96528830464,"internally_reserved":0,"allocated":69976064,"data_stored":82697790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":37737738},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63760,"kb_used_data":24264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308080,"statfs":{"total":96636764160,"available":96571473920,"internally_reserved":0,"allocated":24846336,"data_stored":27480392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86760,"kb_used_data":41240,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94285080,"statfs":{"total":96636764160,"available":96547921920,"internally_reserved":0,"allocated":42229760,"data_stored":50882979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102320,"kb_used_data":58660,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94269520,"statfs":{"total":96636764160,"available":96531988480,"internally_reserved":0,"allocated":60067840,"data_stored":74386784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41836,"kb_used_data":10652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330004,"statfs":{"total":96636764160,"available":96593924096,"internally_reserved":0,"allocated":10907648,"data_stored":11450373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123992,"kb_used_data":80332,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94247848,"statfs":{"total":96636764160,"available":96509796352,"internally_reserved":0,"allocated":82259968,"data_stored":101234194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66520,"kb_used_data":39500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94305320,"statfs":{"total":96636764160,"available":96568647680,"internally_reserved":0,"allocated":40448000,"data_stored":51073316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42528,"kb_used_data":11348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329312,"statfs":{"total":96636764160,"available":96593215488,"internally_reserved":0,"allocated":11620352,"data_stored":14174716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40706048,"data_stored":50541567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23920640,"data_stored":27182832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8548352,"data_stored":11630274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69050368,"data_stored":82280502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":42426368,"data_stored":53540795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81391616,"data_stored":100860457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11493376,"data_stored":12521226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61358080,"data_stored":76171341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:51:05.270 INFO:tasks.thrashosds.thrasher:Setting 3.14 to [1, 4] 2024-04-23T23:51:05.271 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.14', '1', '4'] 2024-04-23T23:51:05.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.14 1 4 2024-04-23T23:51:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:51:05.343 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:05.357+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:05.350 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:05.350 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:05.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:05.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:05.444 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:05.459+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:05.514 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:05.514 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:05.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:05.528 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:05.545 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:05.559+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:05.547 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:05.548 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:05.566 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:05.566 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:05.583 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:05.601 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:05.618 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:05.618 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:05.629 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:05.629 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:05.629 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:05.629 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:05.629 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:05.636 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:05.636 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:05.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:05.653 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:05.654 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:05.654 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:05.667+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:05.747 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:05.760+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.310797844999996, 2024-04-23T23:51:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:5561 3.0 e34/22 3:00530cdb:::rbd_data.1179c4f11774.0000000000000001:head v 34'11, mlcod=34'9)", 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.563138885000001, 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033873340000000001, 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207221+0000", 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207223+0000", 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-06 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207232+0000", 2024-04-23T23:51:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9090000000000006e-06 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207249+0000", 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7748999999999999e-05 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207259+0000", 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4800000000000007e-06 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207314+0000", 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5331000000000003e-05 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.207337+0000", 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2748999999999999e-05 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210565+0000", 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032283780000000001 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:03.210608+0000", 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3183000000000003e-05 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:6657 3.0 e36/22 3:0475559e:::rbd_data.117967c411b0.0000000000000020:head v 36'27, mlcod=36'25)", 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.721269579999998, 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034412309999999999, 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049090+0000", 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049091+0000", 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049097+0000", 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4809999999999997e-06 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049108+0000", 2024-04-23T23:51:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1868000000000001e-05 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049115+0000", 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049165+0000", 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0946000000000003e-05 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.049183+0000", 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8009999999999999e-05 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052497+0000", 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003313524 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:06.052532+0000", 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4669000000000003e-05 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:58072 3.9 e54/22 3:967d967a:::rbd_data.117966a158ce.0000000000000002:head v 54'280, mlcod=53'276)", 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.576912718000003, 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003273686, 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193447+0000", 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193448+0000", 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193458+0000", 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0628000000000001e-05 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193469+0000", 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0380000000000001e-05 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193479+0000", 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193529+0000", 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-05 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.193551+0000", 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196689+0000", 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031378420000000001 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:30.196721+0000", 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2400000000000001e-05 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.501291349999999, 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.451496463000002, 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.450365711, 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:51:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:51:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.404871395000001, 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.402101021, 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63748 3.4 e58/22 3:21ed4642:::rbd_data.11798ae5b3f1.0000000000000064:head v 58'132, mlcod=58'128)", 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.373709999999999, 2024-04-23T23:51:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003193577, 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396650+0000", 2024-04-23T23:51:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396651+0000", 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396657+0000", 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8980000000000001e-06 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396667+0000", 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0288e-05 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396673+0000", 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396718+0000", 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.396736+0000", 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8071e-05 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399806+0000", 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003069881 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.399844+0000", 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8056999999999998e-05 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.306715191999999, 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.30429711, 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64292 3.9 e58/22 3:957b2643:::rbd_data.11798ae5b3f1.000000000000011d:head v 58'325, mlcod=58'323)", 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.240819592000001, 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003216596, 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529540+0000", 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529541+0000", 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529546+0000", 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6619999999999996e-06 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529557+0000", 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1035e-05 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529564+0000", 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0219999999999999e-06 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529615+0000", 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0899000000000002e-05 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.529636+0000", 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1739999999999999e-05 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532716+0000", 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003079755 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.532757+0000", 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.087e-05 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64347 3.4 e58/22 3:23ee6249:::rbd_data.11798ae5b3f1.000000000000013a:head v 58'151, mlcod=58'149)", 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.222006999000001, 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036492059999999999, 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548353+0000", 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548354+0000", 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6899999999999996e-07 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548359+0000", 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548372+0000", 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3050999999999999e-05 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548379+0000", 2024-04-23T23:51:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548448+0000", 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9326999999999995e-05 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548466+0000", 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659000000000002e-05 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551946+0000", 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034799850000000001 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552002+0000", 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6508999999999997e-05 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64341 3.0 e58/22 3:0635a7b0:::rbd_data.11798ae5b3f1.0000000000000138:head v 58'169, mlcod=58'166)", 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.221599095999998, 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032676770000000001, 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548761+0000", 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548766+0000", 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548777+0000", 2024-04-23T23:51:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1453e-05 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548784+0000", 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6460000000000004e-06 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548833+0000", 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8498000000000002e-05 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.548852+0000", 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9142000000000001e-05 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.551992+0000", 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031404649999999998 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.552029+0000", 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6381999999999998e-05 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.214004727999999, 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:51:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:67439 3.0 e58/22 3:0718c4b2:::rbd_data.117911508d31.0000000000000006:head v 58'177, mlcod=58'175)", 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.861236165000001, 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036877949999999998, 2024-04-23T23:51:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909124+0000", 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909125+0000", 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909535+0000", 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040998899999999998 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909550+0000", 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5092000000000001e-05 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909558+0000", 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6790000000000005e-06 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909610+0000", 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1996000000000002e-05 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.909638+0000", 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8231000000000001e-05 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912774+0000", 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031359740000000001 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.912812+0000", 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7710999999999999e-05 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:98054 3.0 e74/22 3:00ee5193:::rbd_data.1179fd9d5edc.0000000000000002:head v 74'235, mlcod=73'233)", 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.20372128, 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003737537, 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.566639+0000", 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1100000000000002e-07 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567428+0000", 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078870700000000004 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567444+0000", 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.626e-05 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567456+0000", 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1902000000000001e-05 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567526+0000", 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0288000000000002e-05 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.567552+0000", 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.622e-05 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.570291+0000", 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002738603 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:53.570376+0000", 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5246000000000003e-05 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.5855498400000005, 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.409966389, 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:05.846 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:05.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:05.848 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:05.862+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:05.948 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:05.962+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:05.949 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:05.949 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:05.949 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:05.949 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:05.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:06.049 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:06.063+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:06.098 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:06.098 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:06.099 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:06.099 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:06.099 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:06.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:06.149 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:06.163+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:06.210 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.792916144000003, 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.792906264000003, 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:06.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:06.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:06.219 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.792695917000003, 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:06.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.790864577999997, 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.788794947, 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276704711000001, 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:06.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:06.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:06.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:06.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276679055000002, 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276611565000003, 2024-04-23T23:51:06.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276566504000002, 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:06.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:06.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:06.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276530004000001, 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276513045000002, 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:06.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276434361, 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276367327999999, 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.276233726999997, 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.275815047000002, 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.275664405000001, 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.275539023, 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.275323505999999, 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.275086172000002, 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.274846277000002, 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.297 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:set 3.14 pg_upmap mapping to [1,4] 2024-04-23T23:51:06.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:06.299 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:06.312+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:06.399 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:06.413+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:06.409 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:06.409 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:06.409 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:06.410 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:06.418 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:06.500 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:06.514+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:06.521 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:06.521 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:06.521 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:06.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:06.602 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:06.616+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:06.674 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:06.674 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.417594268999999, 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:06.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.417573388999998, 2024-04-23T23:51:06.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:06.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.411715602999998, 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.684 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:06.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:06.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.411701356000002, 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:06.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.409667423000002, 2024-04-23T23:51:06.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:06.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.409655481000001, 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.407678347000001, 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.407650263000001, 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:06.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.407637727999997, 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:06.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.405696640000002, 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.714 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.405685294000001, 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.718 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.722 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.405674439999999, 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.405663148999999, 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.727 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.731 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.403624376000003, 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.735 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.403608392999999, 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.736 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:06.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.403587774000002, 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:06.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:06.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.403567776000003, 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.403549495999997, 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:06.753 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.402547298000002, 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:06.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.402535798999999, 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.758 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:06.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:06.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.762 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.762 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:06.762 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:06.762 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:06.766 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:06.777+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:06.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:06.864 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:06.878+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:06.869 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:06.869 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:06.869 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:06.870 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:06.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:06.965 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:06.979+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:07.021 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:07.021 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:07.021 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:07.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:07.070 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:07.084+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:07.166 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:07.180+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:07.174 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.667492869999997, 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:07.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.176 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:07.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:07.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:07.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:07.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:07.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:07.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.667459821999998, 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:07.182 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:07.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.667437323000001, 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:07.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:07.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:07.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:07.191 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.665567406000001, 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:07.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:07.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:07.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:07.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:07.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:07.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:07.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:07.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:07.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:07.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.200 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.200 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.665538929999997, 2024-04-23T23:51:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:07.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.202 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:07.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:07.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:07.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:07.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:07.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:07.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:07.209 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.665516486000001, 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:07.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:07.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:07.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.214 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.663560758000003, 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:07.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:07.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.663541631000001, 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.663515482999998, 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:07.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.663494790999998, 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.228 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:07.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.663473064999998, 2024-04-23T23:51:07.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:07.235 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:07.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:07.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:07.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:07.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:07.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:07.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:07.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:07.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:07.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:07.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:07.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:07.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:07.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:07.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:07.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.661576175, 2024-04-23T23:51:07.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:07.245 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:07.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:07.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:07.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:07.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:07.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:07.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:07.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:07.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:07.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:07.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:07.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:07.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:07.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:07.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:07.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:07.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.254 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.661551727999999, 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:07.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:07.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:07.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.661534633999999, 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:07.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:07.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.659557253000003, 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:07.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:07.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:07.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.659538374999997, 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.271 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:07.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:07.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.657557335, 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:07.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.279 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.657517533000004, 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:07.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.657495509, 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:07.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:07.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.657473967999998, 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:07.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:07.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:07.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.292 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:07.293 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:07.307+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:07.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:07.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:51:07.394 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:07.408+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:07.407 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:07.407 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:07.407 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:07.408 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:07.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:07.494 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:07.508+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:07.517 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:07.518 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:07.518 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:07.518 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:07.518 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:07.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:07.596 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:07.610+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:07.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.481800762000006, 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:07.670 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:07.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.677 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.133064093999998, 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.678 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.133047640999997, 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:07.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:07.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.133020899999998, 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:07.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.133002105000003, 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.698 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.132984520999997, 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:07.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.121289017000002, 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:07.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:07.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.12111333, 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:07.709 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:07.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:07.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:07.714 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.120961811000001, 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:07.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:07.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.120679185, 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:07.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:07.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.724 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.12053332, 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:07.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.119583636000002, 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.734 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.119560741999997, 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.103545367000002, 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.103535733000001, 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.088503371999998, 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.750 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:07.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:07.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.088482048000003, 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.755 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:07.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.088459223000001, 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:07.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:07.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.088436543999997, 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:07.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:07.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:07.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 60.086134821000002, 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:07.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:07.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:07.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:07.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:07.777 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:07.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:07.779 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:07.792+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:07.790 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:07.790 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:07.806 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:07.806 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:07.822 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:07.823 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:07.839 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:07.840 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:07.857 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:07.874 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:07.874 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:07.879 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:07.892+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:07.884 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:07.884 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:07.891 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:07.892 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:07.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:07.910 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:07.980 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:07.995+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:08.012 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.012 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:08.081 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:08.095+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:08.124 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.124 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:08.182 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:08.196+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:08.276 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.276 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.283 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:08.298+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:08.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:51:08.385 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:08.398+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:08.389 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.389 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:08.486 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:08.500+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:08.541 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.541 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:08.587 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:08.601+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:08.688 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:08.702+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:08.708 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.709 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:08.801 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:08.815+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:08.860 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:08.861 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:08.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:08.890 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:08.905+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:08.991 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:09.005+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:09.014 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.014 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:09.092 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:09.106+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:09.123 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.123 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:09.218 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:09.230+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:09.275 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.275 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:09.325 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:09.339+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:09.395 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:09.409+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:09.435 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.436 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:09.496 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:09.509+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:09.547 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.548 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:09.596 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:09.610+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:09.658 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.658 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.667 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:09.710 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:09.723+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:09.770 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.770 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:09.798 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:09.812+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:09.899 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:09.913+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:09.921 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:09.922 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:09.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:09.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:10.000 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:10.013+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:10.034 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:10.034 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:10.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:10.044 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:10.061 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:10.061 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:10.078 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:10.078 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:10.094 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:10.101 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:10.114+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:10.112 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:10.130 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:10.147 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:10.149 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:10.149 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:10.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:10.149 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:10.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.892119616999999, 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.150 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:10.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.892098736999998, 2024-04-23T23:51:10.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.886240950999998, 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.159 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:10.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:10.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.886226704000002, 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:10.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:10.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.884192771000002, 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.168 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.884180829000002, 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.882203695000001, 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.882175611000001, 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.882163075999998, 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.880221988000002, 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.880210642000002, 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.880199787999999, 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.880188496999999, 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.878149723999996, 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.878133740999999, 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:10.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:10.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.878113122000002, 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:10.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:10.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:10.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.878093124000003, 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:10.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.878074843999997, 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:10.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:10.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.877072646000002, 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:10.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.877061146999999, 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:10.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:10.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.240 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:10.241 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:10.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:10.243 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:10.255+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:10.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:51:10.342 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:10.355+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:10.396 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:10.396 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:10.397 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:10.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:10.443 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:10.456+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:10.544 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:10.558+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:10.548 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:10.549 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:10.549 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:10.549 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:10.549 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:10.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:10.646 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:10.661+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:10.661 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:10.661 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:10.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:10.661 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:10.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.155407273000002, 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:10.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:10.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.155374225000003, 2024-04-23T23:51:10.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:10.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.155351725999999, 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:10.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:10.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:10.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:10.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.153481808999999, 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:10.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:10.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:10.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.153453333000002, 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:10.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.153430888999999, 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:10.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.689 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.151475161, 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:10.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:10.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.151456033999999, 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.694 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:10.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:10.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.151429886000003, 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:10.698 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:10.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:10.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.151409194000003, 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.703 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:10.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.151387468000003, 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.149490577999998, 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:10.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.149466130999997, 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:10.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:10.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:10.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.149449036999997, 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:10.720 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.724 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.147471656, 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:10.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:10.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.147452778000002, 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:10.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:10.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.145471737999998, 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:10.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.737 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.145431936000001, 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.741 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.145409911999998, 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:10.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.145388371000003, 2024-04-23T23:51:10.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:10.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:10.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:10.751 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:10.751 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:10.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:10.753 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:10.766+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:10.853 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:10.867+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:10.855 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:10.855 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:10.855 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:10.855 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:10.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:10.954 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:10.968+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:10.964 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:10.964 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:10.964 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:10.965 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:10.965 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:10.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:11.059 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:11.072+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 93.888998790000002, 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.540262122000001, 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.088 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.540245669000001, 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.089 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:11.093 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.540218928000002, 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:11.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.540200132999999, 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.540182549000001, 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.106 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.528487044999999, 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:11.110 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.528311358000003, 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.111 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:11.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.528159838999997, 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.115 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:11.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.527877212999996, 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:11.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:11.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:11.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.527731348000003, 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:11.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:11.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.526781663999998, 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:11.128 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:11.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:11.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.526758770000001, 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:11.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:11.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:11.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.510743394999999, 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:11.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:11.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.510733760999997, 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:11.141 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:11.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:11.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.495701400000002, 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.146 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:11.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:11.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.495680075999999, 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.150 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:11.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.495657250999997, 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.155 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:11.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.495634572, 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:11.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:11.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:11.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.493332848999998, 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:11.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:11.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:11.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:11.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:11.169 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:11.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:11.171 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:11.184+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:11.272 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:11.285+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:11.276 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:11.276 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:11.276 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:11.277 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:11.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:11.312 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:11.312 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:11.312 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-04-23T23:51:11.312 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-23T23:51:11.312 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-23T23:51:11.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-04-23T23:51:11.322 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:51:11.373 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:11.386+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:11.429 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:11.429 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-23T23:51:11.429 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:11.439 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:11.439 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:11.439 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:11.439 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:11.439 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:11.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:11.481 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:11.494+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:11.552 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:11.552 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:11.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.450803473000001, 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:11.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.448209142000003, 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336553703, 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:11.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336535263000002, 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336516156999998, 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336497647999998, 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:11.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336394599000002, 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:11.582 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.586 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336374980000002, 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:11.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336269504999997, 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:11.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:11.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336244319999999, 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:11.595 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336210774999998, 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:11.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336187012000003, 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:11.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336165971, 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.608 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:11.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:11.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336097735999999, 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.612 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:11.616 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336073268, 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:11.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:11.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336047569999998, 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:11.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336025804000002, 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.625 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:11.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:11.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:11.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.336005338, 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:11.629 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:11.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.335927478999999, 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:11.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:11.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.335897828, 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:11.642 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:11.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:11.644 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:11.656+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:11.745 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:11.757+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:11.746 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:11.747 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:11.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:11.845 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:11.859+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:11.858 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:11.858 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:11.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:11.946 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:11.959+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.207969812999998, 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.205925829999998, 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.203948582000002, 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.203917949000001, 2024-04-23T23:51:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.201975099000002, 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.199889890000001, 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.197880458999997, 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.195980491, 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:51:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:51:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.191931963000002, 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.189887286999998, 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.187813714000001, 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.185925558000001, 2024-04-23T23:51:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.185904780999998, 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.177913287000003, 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.175978305000001, 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.175952209999998, 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.794234746000001, 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:51:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:51:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.794192326999998, 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.794156370000003, 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.794132011999999, 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:12.100 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:12.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:12.102 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:12.115+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:12.202 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:12.216+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:12.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:12.245 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:12.246 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:12.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:12.303 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:12.316+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:12.372 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:12.372 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:12.392 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:12.392 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:12.403 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:12.416+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:12.412 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:12.412 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:12.414 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:12.414 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:12.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:12.431 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:12.451 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:12.468 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:12.469 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:12.486 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:12.503 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:12.504 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:12.518+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:12.528 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:12.529 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:12.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:12.622 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:12.636+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:12.681 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:12.681 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:12.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:12.707 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:12.719+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:12.806 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:12.820+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:12.832 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:12.832 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:12.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:12.908 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:12.922+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:12.947 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:12.947 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:12.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:13.008 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:13.021+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:13.074 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.074 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:13.112 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:13.126+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:13.213 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:13.226+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:13.225 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.225 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.234 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:13.319 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:13.332+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:13.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:51:13.380 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.380 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:13.430 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:13.443+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:13.519 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:13.532+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:13.531 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.531 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:13.620 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:13.634+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:13.683 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.683 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.692 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:13.721 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:13.735+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:13.795 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.796 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:13.848 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:13.861+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:13.906 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:13.906 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:13.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:13.922 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:13.936+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:51:14.024 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:14.037+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:14.064 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:14.065 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:14.074 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:14.125 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:14.139+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:14.217 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:14.217 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:14.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:14.226 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:14.238+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:14.327 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:14.341+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:14.369 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:14.369 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:14.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:14.428 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:14.441+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:14.480 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:14.480 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:14.489 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:14.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:14.559 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:14.573+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:14.630 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:14.644+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:14.632 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:14.633 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:14.633 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:14.633 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:14.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:14.651 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:14.670 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:14.670 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:14.700 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:14.717 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:14.717 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:14.730 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:14.744+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:14.735 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:14.735 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:14.752 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:14.753 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:14.756 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:14.756 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:14.756 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:14.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:14.770 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:14.770 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:14.832 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:14.845+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:14.872 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:14.873 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:14.873 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:14.873 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:14.873 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:14.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:14.933 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:14.946+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:14.995 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.893767646999997, 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.891173316, 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779517877000004, 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779499436999998, 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779480331000002, 2024-04-23T23:51:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779461822000002, 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779358772999998, 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779339153999999, 2024-04-23T23:51:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779233679000001, 2024-04-23T23:51:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779208494000002, 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779174949000002, 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779151186, 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779130145000003, 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779061910000003, 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779037442000003, 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.779011744000002, 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.068 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:15.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.778989977999998, 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:15.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.778969512000003, 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:15.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:15.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:15.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.778891653000002, 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:15.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.778862001999997, 2024-04-23T23:51:15.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:15.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:15.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:15.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.091 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:15.092 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:15.105+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:15.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:15.193 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:15.207+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:15.196 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:15.196 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:15.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:15.294 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:15.308+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:15.348 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:15.348 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:15.348 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:15.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:15.400 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:15.414+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:15.496 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:15.510+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:15.502 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.699698202999997, 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:15.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:15.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.697654219999997, 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:15.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:15.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:15.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:15.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:15.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:15.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:15.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:15.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:15.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:15.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:15.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:15.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.695676972000001, 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.521 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.695646339, 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:15.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.693703489000001, 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:15.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.69161828, 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.689608849000003, 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:15.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:15.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:15.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.687708880999999, 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:51:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.683660353, 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:15.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.681615677000003, 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.550 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:15.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:15.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.679542103999999, 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:15.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:15.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:15.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.562 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.677653948, 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.563 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.677633170999997, 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.669641677000001, 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.667706695, 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.667680599999997, 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.285963135999999, 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.285920717000003, 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.285884760000002, 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:15.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.285860401999997, 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.603 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:15.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:15.606 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:15.618+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:15.705 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:15.719+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:15.748 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:15.749 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:15.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:15.806 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:15.820+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:15.858 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:15.858 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:15.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:15.911 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:15.925+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:15.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:51:15.990 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:15.990 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:15.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:15.990 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:15.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.490053707000001, 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.489587720000003, 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:15.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:15.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.489081171000002, 2024-04-23T23:51:16.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.488785262999997, 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.005 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:16.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:16.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.488598250000003, 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:16.014 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.48839658, 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:16.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.488256317999998, 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.488010582999998, 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:16.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.487733058000003, 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.487490485999999, 2024-04-23T23:51:16.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:16.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.487312121999999, 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:16.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:16.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.487029530999997, 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:16.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.486837047000002, 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:16.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:16.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:16.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.486409105999996, 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:16.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.486256773000001, 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:16.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.486153690000002, 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.485836743999997, 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:16.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.485595316999998, 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:16.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:16.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.485370770000003, 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:16.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.485115872999998, 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:16.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.081 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:16.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:16.083 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:16.096+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:16.184 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:16.197+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:16.228 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:16.228 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:16.237 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:16.284 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:16.298+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:16.339 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:16.339 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:16.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:16.385 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:16.399+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:16.441 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:16.441 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:16.441 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-04-23T23:51:16.441 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-23T23:51:16.441 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-23T23:51:16.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-04-23T23:51:16.485 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:16.499+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:16.494 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 75.047685721999997, 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.238179227000003, 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.188384339999999, 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.187253587999997, 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:51:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:51:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.141759272000002, 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.138988898000001, 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.043603069, 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.041184987000001, 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.950892605, 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:51:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.322437717, 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:16.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.146854265999998, 2024-04-23T23:51:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:16.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1451 3.9 e95/22 3:9566a16b:::rbd_data.117911508d31.0000000000000023:1c v 95'434, mlcod=91'429)", 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.5486032010000002, 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043290560000000004, 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.68e-07 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958651+0000", 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958664+0000", 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3604000000000001e-05 2024-04-23T23:51:16.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958671+0000", 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958719+0000", 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7568000000000003e-05 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958742+0000", 2024-04-23T23:51:16.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3532000000000001e-05 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962919+0000", 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041768550000000002 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962974+0000", 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4691999999999997e-05 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1371 3.4 e95/22 3:21d77a66:::rbd_data.117911508d31.000000000000000c:1c v 95'391, mlcod=91'388)", 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.548199229, 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039556620000000004, 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:16.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6599999999999995e-07 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959066+0000", 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6498e-05 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959078+0000", 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493999999999999e-05 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959085+0000", 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959139+0000", 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4214e-05 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959159+0000", 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.931e-05 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962950+0000", 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003791516 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.963004+0000", 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4262999999999999e-05 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1375 3.0 e96/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:1b v 96'269, mlcod=95'264)", 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:16.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54698382800000001, 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054529929999999997, 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960265+0000", 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.61e-07 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960271+0000", 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2750000000000001e-06 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960282+0000", 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056000000000001e-05 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960289+0000", 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7660000000000001e-06 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960552+0000", 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026309199999999998 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960571+0000", 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8980000000000001e-05 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965676+0000", 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051052739999999999 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965717+0000", 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0788999999999999e-05 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1482 3.9 e96/22 3:90d97e73:::rbd_data.11798ae5b3f1.0000000000000008:1b v 96'436, mlcod=95'432)", 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54288527900000005, 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045767280000000004, 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964368+0000", 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964386+0000", 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7876e-05 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964392+0000", 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964440+0000", 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7321999999999999e-05 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964459+0000", 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9313e-05 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968889+0000", 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004429962 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968939+0000", 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0501000000000003e-05 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1378 3.0 e96/22 3:03a7baf4:::rbd_data.11798ae5b3f1.000000000000024d:1b v 96'271, mlcod=96'267)", 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.54052223399999999, 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007336375, 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966731+0000", 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.011e-06 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966740+0000", 2024-04-23T23:51:16.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9579999999999996e-06 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966745+0000", 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966788+0000", 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985000000000003e-05 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966803+0000", 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974010+0000", 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072070320000000004 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974062+0000", 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1805999999999998e-05 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1483 3.9 e96/22 3:94dae484:::rbd_data.11798ae5b3f1.000000000000006c:1b v 96'438, mlcod=95'432)", 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.53959220600000002, 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089730109999999995, 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:16.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967662+0000", 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967675+0000", 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2763e-05 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967683+0000", 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5530000000000004e-06 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967734+0000", 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967756+0000", 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1906999999999999e-05 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976580+0000", 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088237190000000007 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976629+0000", 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8948e-05 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1486 3.9 e96/22 3:96d7965c:::rbd_data.11798ae5b3f1.00000000000003a1:1b v 96'442, mlcod=96'436)", 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.52714819199999996, 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092063349999999995, 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980107+0000", 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4180000000000002e-06 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980117+0000", 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0565e-05 2024-04-23T23:51:16.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980124+0000", 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980185+0000", 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980206+0000", 2024-04-23T23:51:16.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1472e-05 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989259+0000", 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090529410000000001 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989306+0000", 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950000000000003e-05 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1383 3.4 e96/22 3:212acb2d:::rbd_data.11798ae5b3f1.0000000000000184:1b v 96'397, mlcod=96'393)", 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.52145685100000005, 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043511569999999996, 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:16.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985792+0000", 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.062e-06 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985801+0000", 2024-04-23T23:51:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985819+0000", 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-05 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985827+0000", 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8599999999999999e-06 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985877+0000", 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790999999999998e-05 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985900+0000", 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2418000000000001e-05 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990100+0000", 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042000819999999999 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990142+0000", 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2469999999999998e-05 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1384 3.4 e96/22 3:25b57ffa:::rbd_data.11798ae5b3f1.000000000000006e:1b v 96'399, mlcod=96'393)", 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.52079843699999995, 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045474490000000003, 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:16.582 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986450+0000", 2024-04-23T23:51:16.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986457+0000", 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9809999999999997e-06 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986473+0000", 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6016e-05 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986481+0000", 2024-04-23T23:51:16.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986530+0000", 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8773000000000002e-05 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986550+0000", 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0347999999999999e-05 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:16.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990945+0000", 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043951370000000003 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990997+0000", 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1674e-05 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:16.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:16.587 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:16.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:16.589 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:16.589 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-23T23:51:16.589 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:16.590 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:16.602+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:16.689 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:16.702+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:16.702 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:16.702 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:16.703 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:16.703 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:16.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:16.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:16.796 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:16.810+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:16.854 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:16.855 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:16.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:16.891 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:16.905+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:16.901 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:16.918 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:16.918 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:16.935 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:16.935 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:16.946 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:51:16.960 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:16.981 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:16.991 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:17.005+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:17.002 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:17.002 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:17.014 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.014 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.022 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:17.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:17.041 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:17.041 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:17.095 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:17.109+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:17.127 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.127 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.135 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:17.193 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:17.207+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:17.236 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.237 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:17.294 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:17.308+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:17.347 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.347 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:17.395 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:17.409+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:17.460 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.460 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:17.513 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:17.527+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:17.570 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.571 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:17.620 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:17.634+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:17.699 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:17.713+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:17.720 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.721 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:17.800 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:17.814+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:17.831 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.832 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.840 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:17.901 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:17.915+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:17.946 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:17.946 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:17.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:18.043 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:18.057+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:18.102 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:18.116+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:18.107 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.107 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:18.203 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:18.217+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:18.220 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.220 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:18.314 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:18.328+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:18.373 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.373 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:18.423 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:18.437+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:18.506 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:18.519+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:18.524 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.524 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:18.606 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:18.620+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:18.676 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.676 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:18.707 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:18.721+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:18.787 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.787 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:18.840 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:18.854+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:18.898 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:18.898 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:18.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:18.949 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:18.962+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:18.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:51:19.035 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:19.049+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:19.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:19.069 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:19.070 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:19.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:19.122 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:19.136+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:19.183 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:19.184 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:19.210 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:19.224+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:19.216 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:19.216 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:19.233 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:19.233 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:19.240 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:19.240 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:19.240 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:19.241 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:19.241 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:19.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:19.250 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:19.251 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:19.268 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:19.268 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:19.286 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:19.303 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:19.311 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:19.325+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:19.336 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:19.369 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:19.369 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:19.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:19.369 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:19.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.369 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.868793205000003, 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:19.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.868327217999997, 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.867820668999997, 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:19.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.867524760999999, 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.867337747999997, 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.867136078000001, 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.866995815999999, 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.866750080999999, 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.866472555999998, 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.866229984, 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.86605162, 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.865769028999999, 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.865576545000003, 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.865148603999998, 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.864996271000003, 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.864893188000003, 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.864576241999998, 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.864334814999999, 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.864110267999997, 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.863855371, 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:19.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:19.479 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:19.492+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:19.579 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:19.593+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:19.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:19.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:19.623 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:19.623 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:19.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:19.680 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:19.694+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:19.774 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:19.775 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:19.775 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:19.775 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:19.775 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:19.780 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:19.794+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:19.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:19.881 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:19.895+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.481340936999999, 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.671834441999998, 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:19.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.622039555000001, 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63238 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'146, mlcod=55'142)", 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.620908802999999, 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040585539999999998, 2024-04-23T23:51:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319994+0000", 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319995+0000", 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320003+0000", 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320019+0000", 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6039999999999999e-05 2024-04-23T23:51:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320027+0000", 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8620000000000003e-06 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320072+0000", 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5556999999999998e-05 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.320093+0000", 2024-04-23T23:51:19.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324011+0000", 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039180040000000001 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.324053+0000", 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-05 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.575414487000003, 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:19.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:19.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.572644113000003, 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:19.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.477258284000001, 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:19.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.474840202000003, 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:19.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64369 3.9 e58/22 3:9286ecb7:::rbd_data.11798ae5b3f1.0000000000000147:head v 58'329, mlcod=58'326)", 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.384547820000002, 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040028950000000002, 2024-04-23T23:51:19.964 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556355+0000", 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556356+0000", 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556361+0000", 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556370+0000", 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4409999999999999e-06 2024-04-23T23:51:19.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556375+0000", 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.634e-06 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556413+0000", 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8109999999999999e-05 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.556424+0000", 2024-04-23T23:51:19.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1362000000000001e-05 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560314+0000", 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038890909999999999 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.560358+0000", 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4638000000000002e-05 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756092932000001, 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:19.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.580509481, 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1451 3.9 e95/22 3:9566a16b:::rbd_data.117911508d31.0000000000000023:1c v 95'434, mlcod=91'429)", 2024-04-23T23:51:19.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.9822584159999996, 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043290560000000004, 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.68e-07 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958651+0000", 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958664+0000", 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3604000000000001e-05 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958671+0000", 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958719+0000", 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7568000000000003e-05 2024-04-23T23:51:19.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958742+0000", 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3532000000000001e-05 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962919+0000", 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041768550000000002 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962974+0000", 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4691999999999997e-05 2024-04-23T23:51:19.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1371 3.4 e95/22 3:21d77a66:::rbd_data.117911508d31.000000000000000c:1c v 95'391, mlcod=91'388)", 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.9818544439999997, 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039556620000000004, 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959049+0000", 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6599999999999995e-07 2024-04-23T23:51:19.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959066+0000", 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6498e-05 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959078+0000", 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493999999999999e-05 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959085+0000", 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959139+0000", 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4214e-05 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.959159+0000", 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.931e-05 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962950+0000", 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003791516 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.963004+0000", 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4262999999999999e-05 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1375 3.0 e96/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:1b v 96'269, mlcod=95'264)", 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.980639043, 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054529929999999997, 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960265+0000", 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.61e-07 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960271+0000", 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2750000000000001e-06 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960282+0000", 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056000000000001e-05 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960289+0000", 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7660000000000001e-06 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960552+0000", 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026309199999999998 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960571+0000", 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8980000000000001e-05 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965676+0000", 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051052739999999999 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965717+0000", 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0788999999999999e-05 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1482 3.9 e96/22 3:90d97e73:::rbd_data.11798ae5b3f1.0000000000000008:1b v 96'436, mlcod=95'432)", 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:19.990 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.976540494, 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045767280000000004, 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:19.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964368+0000", 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964386+0000", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7876e-05 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964392+0000", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964440+0000", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7321999999999999e-05 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964459+0000", 2024-04-23T23:51:19.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9313e-05 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968889+0000", 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004429962 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968939+0000", 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0501000000000003e-05 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1378 3.0 e96/22 3:03a7baf4:::rbd_data.11798ae5b3f1.000000000000024d:1b v 96'271, mlcod=96'267)", 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9741774489999999, 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007336375, 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966731+0000", 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.011e-06 2024-04-23T23:51:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966740+0000", 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9579999999999996e-06 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966745+0000", 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966788+0000", 2024-04-23T23:51:19.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985000000000003e-05 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966803+0000", 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974010+0000", 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072070320000000004 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974062+0000", 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1805999999999998e-05 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1483 3.9 e96/22 3:94dae484:::rbd_data.11798ae5b3f1.000000000000006c:1b v 96'438, mlcod=95'432)", 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9732474209999999, 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089730109999999995, 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:20.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967662+0000", 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967675+0000", 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2763e-05 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967683+0000", 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5530000000000004e-06 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967734+0000", 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967756+0000", 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1906999999999999e-05 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976580+0000", 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088237190000000007 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976629+0000", 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8948e-05 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1486 3.9 e96/22 3:96d7965c:::rbd_data.11798ae5b3f1.00000000000003a1:1b v 96'442, mlcod=96'436)", 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9608034070000002, 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092063349999999995, 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:51:20.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980107+0000", 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4180000000000002e-06 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980117+0000", 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0565e-05 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980124+0000", 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980185+0000", 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980206+0000", 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1472e-05 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989259+0000", 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090529410000000001 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989306+0000", 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950000000000003e-05 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1383 3.4 e96/22 3:212acb2d:::rbd_data.11798ae5b3f1.0000000000000184:1b v 96'397, mlcod=96'393)", 2024-04-23T23:51:20.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9551120659999999, 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043511569999999996, 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985792+0000", 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.062e-06 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985801+0000", 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985819+0000", 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-05 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985827+0000", 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8599999999999999e-06 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985877+0000", 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790999999999998e-05 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985900+0000", 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2418000000000001e-05 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990100+0000", 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042000819999999999 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990142+0000", 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2469999999999998e-05 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.012 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1384 3.4 e96/22 3:25b57ffa:::rbd_data.11798ae5b3f1.000000000000006e:1b v 96'399, mlcod=96'393)", 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9544536519999998, 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045474490000000003, 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:20.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986450+0000", 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:20.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986457+0000", 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9809999999999997e-06 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986473+0000", 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6016e-05 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986481+0000", 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986530+0000", 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8773000000000002e-05 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986550+0000", 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0347999999999999e-05 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990945+0000", 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043951370000000003 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:20.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990997+0000", 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1674e-05 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:20.017 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:20.018 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:20.032+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:20.019 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:51:20.119 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:20.133+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:20.131 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:20.131 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:20.131 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:20.131 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:20.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:20.221 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:20.235+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:20.240 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:20.240 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:20.240 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:20.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:20.322 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:20.336+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:20.352 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:20.352 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.934458221, 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.353 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:20.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:20.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.934448341, 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.358 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:20.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.934237994, 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:20.363 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:20.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.932406655000001, 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:20.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:20.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.930337023999996, 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:20.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.379 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.418246787999998, 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:20.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.418221131999999, 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:20.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:20.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.390 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.418153642, 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:20.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:20.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.418108580999998, 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:20.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.418072080999998, 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.418055121999998, 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.417976437999997, 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.417909405000003, 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.417775804000001, 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.417357123999999, 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.417206481999997, 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.417081099999997, 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.416865583000003, 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.416628248999999, 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.416388353999999, 2024-04-23T23:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.459 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:20.461 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:20.461 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:20.474+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:20.561 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:20.575+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:20.604 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:20.605 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:20.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:20.663 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:20.677+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:20.714 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:20.714 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:20.714 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:20.714 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:20.714 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:20.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:20.763 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:20.777+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:20.825 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:20.825 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:20.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.568533629999997, 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.568512749999996, 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.830 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:20.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:20.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:20.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.562654964000004, 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.835 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:20.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:20.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:20.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.562640717000001, 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:20.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:20.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.560606784000001, 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:20.843 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.560594842, 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.558617708, 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.558589624, 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.558577089000003, 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.556636001000001, 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.556624655, 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.556613800999997, 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.556602509999998, 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.554563737000002, 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.554547753999998, 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.554527135000001, 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.554507137000002, 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.554488857000003, 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.553486659000001, 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.553475159999998, 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:20.913 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:20.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:20.915 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:20.928+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:21.015 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:21.029+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:21.027 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:21.028 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:21.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:21.122 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:21.136+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:21.180 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:21.181 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:21.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:21.216 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:21.231+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:21.317 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:21.331+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:21.332 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.826928742, 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.826895694000001, 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.826873194999997, 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.825003277999997, 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.824974802, 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.824952357999997, 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.822996629999999, 2024-04-23T23:51:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.822977502999997, 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.822951355000001, 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.822930663000001, 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.822908937000001, 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.821012046999996, 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.820987600000002, 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.820970506000002, 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.818993124999999, 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.818974247, 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.816993207000003, 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.816953405, 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.816931381000003, 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.816909840000001, 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:21.422 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:21.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:21.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:21.425 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:21.438+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:21.525 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:21.539+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:21.549 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:21.566 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:21.567 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:21.568 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:21.569 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:21.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:21.583 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:21.583 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:21.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:21.590 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:21.591 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-04-23T23:51:21.591 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-23T23:51:21.591 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-23T23:51:21.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-04-23T23:51:21.600 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:21.600 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:21.617 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:21.617 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:21.634 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:21.650 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:21.651 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:21.663 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:21.677+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:21.669 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:21.669 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:21.683 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:21.683 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:21.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:21.725 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "" 2024-04-23T23:51:21.725 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:21.727 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:21.741+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:21.826 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:21.827 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:21.828 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:21.842+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:21.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:21.929 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:21.943+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:21.938 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:21.938 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:21.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:22.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:51:22.064 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:22.078+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:22.064 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.064 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:22.159 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:22.173+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:22.221 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.221 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:22.232 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:22.246+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:22.333 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:22.346+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:22.374 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.374 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:22.434 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:22.448+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:22.525 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.525 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:22.535 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:22.549+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:22.636 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.636 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.637 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:22.650+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:22.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:22.737 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:22.751+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:22.839 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:22.853+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:22.939 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:51:22.940 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:22.953+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:22.946 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:22.946 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:22.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:23.043 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:23.057+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:23.106 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:23.106 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:23.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:23.142 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:23.156+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:23.243 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:23.257+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:23.257 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:23.257 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:23.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:23.343 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:23.357+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:23.368 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:23.368 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:23.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:23.444 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:23.458+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:23.521 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:23.522 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:23.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:23.545 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:23.559+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:23.645 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:23.646 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:23.646 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:23.660+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:23.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:23.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:23.747 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:23.761+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:23.800 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:23.800 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:23.800 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:23.801 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:23.809 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:23.817 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:23.834 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:23.850 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:23.864+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:23.850 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:23.867 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:23.867 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:23.885 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:23.885 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:23.901 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:23.920 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:23.920 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:23.949 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:23.963+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.536658520000003, 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.536648640000003, 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.536438293000003, 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.534606953999997, 2024-04-23T23:51:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.532537323, 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020447087000001, 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020421431000003, 2024-04-23T23:51:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020353941000003, 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020308880000002, 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020272380000002, 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020255421000002, 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020176737, 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.020109703999999, 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.019976102999998, 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.019557423000002, 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.019406781000001, 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.019281399, 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.019065882, 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.018828548000002, 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.018588653000002, 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:24.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:24.056 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:24.069+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:24.156 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:24.170+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:24.198 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:24.198 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:24.198 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:24.198 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:24.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:24.257 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:24.271+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:24.307 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:24.307 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:24.308 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:24.308 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:24.308 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:24.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:24.358 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:24.372+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:24.419 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.162650536000001, 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:24.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:24.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.162629656, 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:24.424 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:24.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:24.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.15677187, 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.156757622999997, 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:24.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:24.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.154723689999997, 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.439 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:24.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.154711747999997, 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.443 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:24.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:24.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.447 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.152734614000003, 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:24.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.152706530000003, 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:24.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.456 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.152693995, 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:24.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:24.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.150752906999998, 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:24.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:24.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.150741560999997, 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:24.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.150730707000001, 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.475 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.150719416000001, 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:24.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:24.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.148680642999999, 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:24.480 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:24.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.484 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.148664660000001, 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:24.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:24.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.148644040999997, 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:24.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.148624042999998, 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:24.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:24.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.148605762999999, 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:24.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:24.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.147603564999997, 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:24.502 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:24.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.147592066000001, 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:24.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.511 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.511 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:24.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:24.513 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:24.526+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:24.613 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:24.627+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:24.654 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:24.655 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:24.655 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:24.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:24.714 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:24.727+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:24.765 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:24.765 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:24.765 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:24.765 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:24.765 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:24.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:24.814 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:24.828+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:24.915 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:24.929+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:24.917 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:24.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.411801943999997, 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.922 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.411768895999998, 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:24.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:24.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.411746397000002, 2024-04-23T23:51:24.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:24.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:24.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.409876480000001, 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:24.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:24.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:24.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.409848003999997, 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:24.936 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:24.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.940 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.409825560000002, 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:24.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:24.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:24.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.407869832000003, 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:24.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:24.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:24.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.407850705000001, 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:24.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.407824556999998, 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:24.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.407803864999998, 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.962 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.407782138999998, 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.966 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.405885249000001, 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:24.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.405860801999999, 2024-04-23T23:51:24.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.405843707999999, 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:24.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:24.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:24.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.403866327000003, 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:24.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:24.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:24.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.403847448999997, 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:24.986 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.401866409, 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:24.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.401826606999997, 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:24.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:24.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.401804583000001, 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:24.999 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:25.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.401783041999998, 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:25.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:25.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.008 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:25.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:51:25.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:25.051 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:25.065+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:25.117 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:25.131+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:25.159 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:25.159 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:25.159 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:25.159 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:25.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:25.218 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:25.232+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:25.311 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:25.311 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:25.311 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:25.318 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:25.332+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:25.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:25.419 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:25.433+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:25.423 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.423 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.235683877, 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.430 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.886947208999999, 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:25.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:25.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.886930755999998, 2024-04-23T23:51:25.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:25.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.886904014999999, 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:25.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:25.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:25.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.886885219999996, 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.886867636000005, 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.875172132000003, 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.874996444999994, 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.463 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.874844925999994, 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:25.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:25.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.874562299999994, 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.468 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:25.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:25.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.874416435000001, 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:25.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.873466750999995, 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:25.477 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:25.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.873443856999998, 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.482 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.857428482000003, 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:25.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:25.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:25.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:25.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.857418847999995, 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:25.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:25.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:25.497 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.842386486999999, 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:25.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.842365162999997, 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:25.503 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:25.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:25.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:25.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.508 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.842342337999995, 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:25.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.513 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.842319658999998, 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.514 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:25.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.518 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.840017935999995, 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:25.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:25.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:25.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:25.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:25.526 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:25.526 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:25.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:25.528 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:25.541+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:25.627 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:25.641+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:25.630 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:25.630 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:25.631 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:25.631 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:25.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:25.729 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:25.742+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:25.741 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:25.741 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:25.741 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:25.741 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:25.741 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:25.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:25.830 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:25.844+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.792196734000001, 2024-04-23T23:51:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.789602403000004, 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.904 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677946964, 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677928524000002, 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:25.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:25.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677909417999999, 2024-04-23T23:51:25.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:25.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:25.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:25.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677890908999998, 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.918 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677787860000002, 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677768241000003, 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677662765999997, 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677637580999999, 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677604035999998, 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677580272999997, 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677559232, 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677490997, 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677466529, 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677440830999998, 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677419065000002, 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677398599, 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677320739999999, 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.677291089000001, 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:25.982 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:25.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:25.984 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:25.998+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:25.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:51:26.084 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:26.098+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:26.141 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:26.141 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:26.141 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:26.141 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:26.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:26.151 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:26.151 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:26.169 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:26.169 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:26.185 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:26.199+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:26.186 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:26.203 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:26.203 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:26.220 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:26.221 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:26.238 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:26.240 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:26.255 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:26.255 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:26.255 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:26.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:26.273 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:26.273 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:26.286 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:26.300+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:26.376 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:26.376 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:26.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:26.387 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:26.401+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:26.488 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:26.502+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:26.488 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:26.488 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:26.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:26.589 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:26.603+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:26.640 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:26.640 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:26.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:26.690 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:26.704+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:26.734 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:26.734 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:26.734 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-23T23:51:26.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-04-23T23:51:26.790 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:26.804+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:26.793 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:26.794 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:26.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:26.891 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:26.904+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:26.948 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:26.948 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:26.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:26.991 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:27.005+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:27.042 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-04-23T23:51:27.054+0000 7f856ad67640 -1 mon.a@0(leader).osd e107 definitely_dead 0 2024-04-23T23:51:27.092 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:27.106+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:27.117 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.117 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:27.193 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:27.206+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:27.229 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.229 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:27.294 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:27.308+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:27.345 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.345 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:27.395 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:27.409+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.496 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:27.509+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:27.504 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:27.597 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:27.611+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:27.605 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.605 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:27.699 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:27.711+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:27.756 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.757 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:27.809 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:27.823+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:27.867 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:27.867 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:27.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:27.899 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:27.913+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.1. 2024-04-23T23:51:28.000 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:28.014+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:28.023 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:28.023 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:28.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:28.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:51:28.117 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:28.131+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:28.176 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:28.176 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:28.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:28.226 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:28.240+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:28.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:28.304 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:28.318+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:28.328 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:28.328 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:28.337 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:28.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:28.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:28.405 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:28.419+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:28.421 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:28.421 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:28.437 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:28.438 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:28.454 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:28.454 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:28.471 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:28.471 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:28.479 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:28.479 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:28.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:28.489 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:28.489 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:28.506 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:28.506 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:28.523 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:28.523 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:28.532 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:28.546+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:28.591 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:28.591 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:28.591 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:28.591 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:28.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:28.641 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:28.655+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:28.708 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:28.722+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:28.743 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:28.743 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:28.743 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:28.743 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:28.743 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:28.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:28.835 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:28.849+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:28.894 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:28.894 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:28.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:28.894 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:28.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.894 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 111.70654694700001, 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:28.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:28.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:28.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.357810279000006, 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.901 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.905 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.357793826000005, 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:28.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:28.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:28.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.357767085000006, 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:28.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:28.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:28.914 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.357748290000004, 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:28.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.918 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.357730705999998, 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:28.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:28.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.346035201999996, 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.923 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.927 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.345859515000001, 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:28.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:28.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.345707996000002, 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:28.932 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:28.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:28.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.345425370000001, 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.345279504999993, 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:28.941 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:28.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.945 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.344329821000002, 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:28.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.344306927000005, 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.950 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:28.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.328291551999996, 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:28.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.328281918000002, 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.963 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.313249557000006, 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:28.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:28.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.313228233000004, 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.968 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:28.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:28.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.972 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.313205408000002, 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:28.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.976 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.313182729000005, 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:28.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.310881006000002, 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:28.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:28.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:28.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:28.987 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:28.987 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:28.989 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:51:28.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:28.989 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:29.002+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:29.089 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:29.102+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:29.093 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:29.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:29.094 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:29.094 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:29.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:29.189 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:29.204+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:29.245 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:29.246 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:29.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:29.298 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:29.312+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:29.358 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:29.358 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:29.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.256535741, 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:29.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:29.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:29.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:29.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.253941410000003, 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142285971, 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142267531000002, 2024-04-23T23:51:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142248424999998, 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142229915999998, 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142126867000002, 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:29.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:29.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142107248000002, 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.142001772999997, 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:29.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:29.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141976587999999, 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:29.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141943042999998, 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:29.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:29.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141919280000003, 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.413 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141898239, 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141830003999999, 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141805536, 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:29.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141779837999998, 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141758072000002, 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:29.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:29.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141737606, 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141659746999999, 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:29.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:29.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.141630096, 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:29.442 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.446 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:29.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:29.448 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:29.461+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:29.549 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:29.562+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:29.592 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:29.592 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:29.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:29.685 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:29.698+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:29.741 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:29.741 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:29.742 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:29.742 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:29.742 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:29.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:29.791 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:29.805+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:29.852 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:29.866+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:29.895 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:29.895 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:29.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:29.895 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:29.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.092487276, 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:29.901 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.090443293000007, 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:29.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:29.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.905 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.088466045000004, 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:29.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.088435411999995, 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:29.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.086492562000004, 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:29.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.084407353000003, 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:29.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:29.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:29.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.082397921999998, 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.927 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.080497953999995, 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:29.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.076449425999996, 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:29.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.936 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.074404749999999, 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:29.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.941 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.072331176999995, 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:29.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:29.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.070443021000003, 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:29.946 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:29.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.070422244, 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:29.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:29.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.062430750000004, 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:29.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.060495767999996, 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:29.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.060469673, 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.678752209000002, 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.678709789999999, 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:29.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.678673832999998, 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:29.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:29.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.678649475, 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:29.981 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:29.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:29.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:29.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:29.985 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:29.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:29.987 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:30.000+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:30.087 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:30.101+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:30.151 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:30.152 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:30.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:30.187 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:30.201+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:30.279 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:30.279 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:30.279 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:30.280 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:30.280 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:30.288 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:30.302+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:30.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:30.389 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:30.403+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:30.435 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.935253208000006, 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.934787220999993, 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.934280672, 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.933984764000002, 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.933797751, 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.933596081000005, 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:30.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:30.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.933455819000002, 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:30.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.933210083999995, 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:30.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:30.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:30.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.932932558999994, 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:30.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.932689987000003, 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:30.485 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.932511622999996, 2024-04-23T23:51:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.932229031999995, 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.932036547999999, 2024-04-23T23:51:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.931608607000001, 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.931456273999999, 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.931353190999999, 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.931036245000001, 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.930794817999995, 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.930570270999993, 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.930315374000003, 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:30.543 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:30.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:30.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:30.546 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:30.558+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:30.645 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:30.659+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:30.690 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:30.691 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:30.691 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:30.691 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:30.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:30.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:30.724 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:30.741 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:30.746 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:30.760+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:30.758 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:30.776 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:30.794 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:30.795 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:30.809 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:30.809 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:30.811 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:30.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:30.828 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:30.848 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:30.862+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:30.938 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:30.949 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:30.962+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:30.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:51:31.047 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.048 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.049 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:31.063+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:31.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:31.150 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:31.164+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:31.158 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.158 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:31.251 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:31.265+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:31.310 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.310 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:31.360 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:31.374+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:31.453 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:31.467+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:31.461 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.462 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.471 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:31.555 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:31.569+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:31.612 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.613 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:31.662 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:31.676+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:31.756 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:31.770+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:31.763 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.763 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:31.856 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:31.870+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:31.873 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:31.873 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:31.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:31.957 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:31.971+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:32.023 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.024 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:32.035 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:51:32.077 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:32.091+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:32.149 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.149 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:32.159 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:32.172+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:32.259 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:32.273+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:32.301 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.302 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.310 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:32.394 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:32.408+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:32.453 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.453 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.461 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:32.475+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:32.462 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:32.562 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:32.576+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:32.604 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.604 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:32.663 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:32.677+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:32.731 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.731 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:32.764 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:32.779+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:32.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:32.847 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:32.847 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:32.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:32.897 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:32.911+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:32.960 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:32.961 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:32.961 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-23T23:51:32.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-23T23:51:32.962 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:32.962 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:32.966 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:32.980+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:32.979 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:33.012 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:33.022 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:33.022 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:33.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:33.034 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:33.035 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:33.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:33.067 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:33.081+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:33.082 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:33.082 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:33.136 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:33.159 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:33.159 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:33.168 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:33.182+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:33.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:33.269 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:33.283+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:51:33.287 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 113, 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "6a51c05a-822c-4999-80c7-e8408cd2a202", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-04-23T23:49:23.155452+0000", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-04-23T23:51:33.046605+0000", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-04-23T23:50:32.112122+0000", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-04-23T23:51:27.949203+0000", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5804032, 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "nodeep-scrub", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-04-23T23:51:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 6, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 3, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "squid", 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_crimson": false, 2024-04-23T23:51:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:35.387984+0000", 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:51:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 1, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 1, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 1, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 1, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 1, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:51:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "18", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:51:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:51:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:51:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 8, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 8, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.25, 2024-04-23T23:51:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 2, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 2, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "rbd", 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:41.909447+0000", 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:51:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 8, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 8, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 8, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 8, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 8, 2024-04-23T23:51:33.297 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "21", 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:51:33.298 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:51:33.299 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:51:33.300 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.9900000095367432, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.9900000095367432, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.87999999523162842, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.75, 2024-04-23T23:51:33.301 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.75, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "pool_client.0", 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-04-23T23:49:49.482932+0000", 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-04-23T23:51:33.302 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-04-23T23:51:33.303 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "113", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-04-23T23:51:33.304 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 40, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 113, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-23T23:51:33.305 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-04-23T23:51:33.306 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.7400000095367432, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.7400000095367432, 2024-04-23T23:51:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.87999999523162842, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.5299999713897705, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.5299999713897705, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 1, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 1 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "6a84eb3e-7d8a-4436-956d-f7da68d06a21", 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 111, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-04-23T23:51:33.309 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-04-23T23:51:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-04-23T23:51:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-04-23T23:51:33.312 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3364976130 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6820/3364976130", 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6824/3364976130", 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3364976130", 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6828/3364976130", 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.313 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "be35c480-9a84-4846-9f0b-043c7c2c9f4a", 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 0, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 0, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-04-23T23:51:33.314 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 89, 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2924722109 2024-04-23T23:51:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6817/2924722109", 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/2924722109", 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6829/2924722109", 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6825/2924722109", 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "64e8bbbb-690a-4267-a030-30ec66f30bfa", 2024-04-23T23:51:33.319 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 111, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-04-23T23:51:33.320 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.323 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3952319845 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6801/3952319845", 2024-04-23T23:51:33.324 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6803/3952319845", 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3952319845", 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3952319845", 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "692beda9-1302-4d96-81ab-62c40a12e3f1", 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 15, 2024-04-23T23:51:33.325 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 38, 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 56, 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 111, 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 39, 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-04-23T23:51:33.327 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.328 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.329 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1499077312 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6809/1499077312", 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6811/1499077312", 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1499077312", 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1499077312", 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.330 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "705f6b1f-c35d-4ed8-a2c9-70b9324691e3", 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 111, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.331 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6824", 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6825", 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6826", 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6827", 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6830", 2024-04-23T23:51:33.333 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6831", 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6828", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6829", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1281396523 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6825/1281396523", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6827/1281396523", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6831/1281396523", 2024-04-23T23:51:33.335 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6829/1281396523", 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "eb37446d-b5c9-4200-a1e9-38c1f6268401", 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.336 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 111, 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6800", 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6801", 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6802", 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6803", 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6806", 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6807", 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6804", 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6805", 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 612886539 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6801/612886539", 2024-04-23T23:51:33.341 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6803/612886539", 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6807/612886539", 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6805/612886539", 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "c38a4da5-1017-4bb9-86ab-999ea57999c9", 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.342 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 108, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6816", 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6817", 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6818", 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6819", 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6822", 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6823", 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6820", 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6821", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2243867625 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6817/2243867625", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6819/2243867625", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6823/2243867625", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6821/2243867625", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "a921d465-920a-4ad0-839f-d97cd5e6ce62", 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 108, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-04-23T23:51:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6808", 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6809", 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6810", 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6811", 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6814", 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6815", 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.351 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6812", 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.119:6813", 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3209176445 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.352 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.119:6809/3209176445", 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.119:6811/3209176445", 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.119:6815/3209176445", 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.119:6813/3209176445", 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.353 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.405502+0000", 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.354 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 65536, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.334094+0000", 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.268468+0000", 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.367807+0000", 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 39 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.356 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.328752+0000", 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.255936+0000", 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.298943+0000", 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-04-23T23:51:33.358 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540701547738038271, 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-04-23T23:49:31.352632+0000", 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [ 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "3.14", 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-04-23T23:51:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: 1, 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: 4 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [], 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_primaries": [], 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": {}, 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-04-23T23:51:33.360 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [ 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "snaps": [ 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "begin": 13, 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: "length": 10 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [ 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: "snaps": [ 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: "begin": 13, 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: "length": 1 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.362 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.363 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:33.365 INFO:tasks.thrashosds.thrasher:j is {'epoch': 113, 'fsid': '6a51c05a-822c-4999-80c7-e8408cd2a202', 'created': '2024-04-23T23:49:23.155452+0000', 'modified': '2024-04-23T23:51:33.046605+0000', 'last_up_change': '2024-04-23T23:50:32.112122+0000', 'last_in_change': '2024-04-23T23:51:27.949203+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T23:49:35.387984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-23T23:49:41.909447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-04-23T23:49:49.482932+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6a84eb3e-7d8a-4436-956d-f7da68d06a21', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6820', 'nonce': 3364976130}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6824', 'nonce': 3364976130}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3364976130}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3364976130}, {'type': 'v1', 'addr': '172.21.15.92:6828', 'nonce': 3364976130}]}, 'public_addr': '172.21.15.92:6820/3364976130', 'cluster_addr': '172.21.15.92:6824/3364976130', 'heartbeat_back_addr': '172.21.15.92:6831/3364976130', 'heartbeat_front_addr': '172.21.15.92:6828/3364976130', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'be35c480-9a84-4846-9f0b-043c7c2c9f4a', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2924722109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6819', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 2924722109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6827', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 2924722109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6823', 'nonce': 2924722109}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 2924722109}]}, 'public_addr': '172.21.15.92:6817/2924722109', 'cluster_addr': '172.21.15.92:6821/2924722109', 'heartbeat_back_addr': '172.21.15.92:6829/2924722109', 'heartbeat_front_addr': '172.21.15.92:6825/2924722109', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64e8bbbb-690a-4267-a030-30ec66f30bfa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3952319845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3952319845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3952319845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3952319845}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3952319845}]}, 'public_addr': '172.21.15.92:6801/3952319845', 'cluster_addr': '172.21.15.92:6803/3952319845', 'heartbeat_back_addr': '172.21.15.92:6807/3952319845', 'heartbeat_front_addr': '172.21.15.92:6805/3952319845', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '692beda9-1302-4d96-81ab-62c40a12e3f1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 38, 'up_from': 56, 'up_thru': 111, 'down_at': 39, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1499077312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1499077312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1499077312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1499077312}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1499077312}]}, 'public_addr': '172.21.15.92:6809/1499077312', 'cluster_addr': '172.21.15.92:6811/1499077312', 'heartbeat_back_addr': '172.21.15.92:6815/1499077312', 'heartbeat_front_addr': '172.21.15.92:6813/1499077312', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '705f6b1f-c35d-4ed8-a2c9-70b9324691e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 1281396523}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 1281396523}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 1281396523}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 1281396523}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 1281396523}]}, 'public_addr': '172.21.15.119:6825/1281396523', 'cluster_addr': '172.21.15.119:6827/1281396523', 'heartbeat_back_addr': '172.21.15.119:6831/1281396523', 'heartbeat_front_addr': '172.21.15.119:6829/1281396523', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb37446d-b5c9-4200-a1e9-38c1f6268401', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 612886539}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 612886539}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 612886539}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 612886539}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 612886539}]}, 'public_addr': '172.21.15.119:6801/612886539', 'cluster_addr': '172.21.15.119:6803/612886539', 'heartbeat_back_addr': '172.21.15.119:6807/612886539', 'heartbeat_front_addr': '172.21.15.119:6805/612886539', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c38a4da5-1017-4bb9-86ab-999ea57999c9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6816', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 2243867625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6819', 'nonce': 2243867625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 2243867625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6820', 'nonce': 2243867625}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 2243867625}]}, 'public_addr': '172.21.15.119:6817/2243867625', 'cluster_addr': '172.21.15.119:6819/2243867625', 'heartbeat_back_addr': '172.21.15.119:6823/2243867625', 'heartbeat_front_addr': '172.21.15.119:6821/2243867625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a921d465-920a-4ad0-839f-d97cd5e6ce62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 3209176445}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6811', 'nonce': 3209176445}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6815', 'nonce': 3209176445}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6812', 'nonce': 3209176445}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 3209176445}]}, 'public_addr': '172.21.15.119:6809/3209176445', 'cluster_addr': '172.21.15.119:6811/3209176445', 'heartbeat_back_addr': '172.21.15.119:6815/3209176445', 'heartbeat_front_addr': '172.21.15.119:6813/3209176445', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.405502+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.334094+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.268468+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.367807+0000', 'dead_epoch': 39}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.328752+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.255936+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.298943+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-23T23:49:31.352632+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.14', 'osds': [1, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 13, 'length': 10}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 13, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-23T23:51:33.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:51:33.366 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:33.366 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:33.366 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:33.367 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:33.367 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:33.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:33.369 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:33.383+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:33.474 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:33.488+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:33.673 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.710026888000002, 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.707982904999994, 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.706005657000006, 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.705975023999997, 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.704032174000005, 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.701946965000005, 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.699937534, 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.698037565999996, 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:51:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.693989037999998, 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.691944362000001, 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.689870788999997, 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.687982633000004, 2024-04-23T23:51:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.687961856000001, 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.679970362000006, 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.678035379999997, 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.678009285000002, 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.296291820999997, 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.296249402000001, 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.296213444999999, 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.296189087000002, 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:33.761 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:33.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:33.762 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:33.687+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:33.763 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:51:33.764 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:51:33.765 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-04-23T23:51:33.376272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601752979,"num_objects":638,"num_object_clones":43,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":638,"num_whiteouts":421,"num_read":153518,"num_read_kb":611696,"num_write":17211,"num_write_kb":305373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":681,"num_bytes_recovered":19769765,"num_keys_recovered":138,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15948,"ondisk_log_size":15948,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":386108,"kb_used_data":117708,"kb_used_omap":20,"kb_used_meta":268267,"kb_avail":660216772,"statfs":{"total":676457349120,"available":676061974528,"internally_reserved":0,"allocated":120532992,"data_stored":145285494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20560,"internal_metadata":274706352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-136527229,"num_objects":-162,"num_object_clones":-64,"num_object_copies":-324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-162,"num_whiteouts":-62,"num_read":630,"num_read_kb":545,"num_write":136,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":694345,"num_keys_recovered":12,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000770"},"pg_stats":[{"pgid":"3.1f","version":"106'633","reported_seq":8551,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-23T23:51:29.956797+0000","last_change":"2024-04-23T23:51:27.959949+0000","last_active":"2024-04-23T23:51:29.956797+0000","last_peered":"2024-04-23T23:51:29.956797+0000","last_clean":"2024-04-23T23:51:29.956797+0000","last_became_active":"2024-04-23T23:49:51.313921+0000","last_became_peered":"2024-04-23T23:49:51.313921+0000","last_unstale":"2024-04-23T23:51:29.956797+0000","last_undegraded":"2024-04-23T23:51:29.956797+0000","last_fullsized":"2024-04-23T23:51:29.956797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:56.535059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057101500000000002,"stat_sum":{"num_bytes":4710521,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":9911,"num_read_kb":23736,"num_write":768,"num_write_kb":10807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.1e","version":"110'787","reported_seq":4063,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T23:51:31.049874+0000","last_change":"2024-04-23T23:51:31.050182+0000","last_active":"2024-04-23T23:51:31.049874+0000","last_peered":"2024-04-23T23:51:31.049874+0000","last_clean":"2024-04-23T23:51:31.049874+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:51:31.049874+0000","last_undegraded":"2024-04-23T23:51:31.049874+0000","last_fullsized":"2024-04-23T23:51:31.049874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:30.101600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00229575,"stat_sum":{"num_bytes":78,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":8,"num_read":5569,"num_read_kb":12676,"num_write":824,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"9"}]},{"pgid":"3.1d","version":"107'376","reported_seq":3169,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959847+0000","last_change":"2024-04-23T23:51:27.960745+0000","last_active":"2024-04-23T23:51:28.959847+0000","last_peered":"2024-04-23T23:51:28.959847+0000","last_clean":"2024-04-23T23:51:28.959847+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:51:28.959847+0000","last_undegraded":"2024-04-23T23:51:28.959847+0000","last_fullsized":"2024-04-23T23:51:28.959847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:17.385469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060715499999999996,"stat_sum":{"num_bytes":1048576,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":378,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.1c","version":"107'722","reported_seq":6137,"reported_epoch":108,"state":"remapped+peering","last_fresh":"2024-04-23T23:51:27.959657+0000","last_change":"2024-04-23T23:51:28.960307+0000","last_active":"2024-04-23T23:51:27.959657+0000","last_peered":"2024-04-23T23:51:27.083744+0000","last_clean":"2024-04-23T23:51:27.083744+0000","last_became_active":"2024-04-23T23:50:42.175291+0000","last_became_peered":"2024-04-23T23:50:42.175291+0000","last_unstale":"2024-04-23T23:51:27.959657+0000","last_undegraded":"2024-04-23T23:51:27.959657+0000","last_fullsized":"2024-04-23T23:51:27.959657+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011483330000000001,"stat_sum":{"num_bytes":18856878,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":10,"num_read":7259,"num_read_kb":30492,"num_write":869,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2296821,"num_keys_recovered":67,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"8"}]},{"pgid":"3.1b","version":"110'274","reported_seq":3692,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-23T23:51:29.957190+0000","last_change":"2024-04-23T23:51:27.958513+0000","last_active":"2024-04-23T23:51:29.957190+0000","last_peered":"2024-04-23T23:51:29.957190+0000","last_clean":"2024-04-23T23:51:29.957190+0000","last_became_active":"2024-04-23T23:49:51.320929+0000","last_became_peered":"2024-04-23T23:49:51.320929+0000","last_unstale":"2024-04-23T23:51:29.957190+0000","last_undegraded":"2024-04-23T23:51:29.957190+0000","last_fullsized":"2024-04-23T23:51:29.957190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:09:03.840978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001784622,"stat_sum":{"num_bytes":35296071,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":3797,"num_read_kb":17143,"num_write":235,"num_write_kb":4989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.1a","version":"107'329","reported_seq":4405,"reported_epoch":108,"state":"remapped+peering","last_fresh":"2024-04-23T23:51:27.953494+0000","last_change":"2024-04-23T23:51:28.954405+0000","last_active":"2024-04-23T23:51:27.953494+0000","last_peered":"2024-04-23T23:51:27.122608+0000","last_clean":"2024-04-23T23:51:27.122608+0000","last_became_active":"2024-04-23T23:49:51.317803+0000","last_became_peered":"2024-04-23T23:49:51.317803+0000","last_unstale":"2024-04-23T23:51:27.953494+0000","last_undegraded":"2024-04-23T23:51:27.953494+0000","last_fullsized":"2024-04-23T23:51:27.953494+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0072653079999999998,"stat_sum":{"num_bytes":2641206,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":4824,"num_read_kb":10482,"num_write":308,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"8"}]},{"pgid":"3.19","version":"110'771","reported_seq":6322,"reported_epoch":111,"state":"peering","last_fresh":"2024-04-23T23:51:31.048209+0000","last_change":"2024-04-23T23:51:31.048209+0000","last_active":"2024-04-23T23:51:31.048209+0000","last_peered":"2024-04-23T23:51:30.311404+0000","last_clean":"2024-04-23T23:51:26.038226+0000","last_became_active":"2024-04-23T23:51:29.982907+0000","last_became_peered":"2024-04-23T23:51:29.982907+0000","last_unstale":"2024-04-23T23:51:31.048209+0000","last_undegraded":"2024-04-23T23:51:31.048209+0000","last_fullsized":"2024-04-23T23:51:31.048209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0070831419999999997,"stat_sum":{"num_bytes":1717016,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":15,"num_read":8097,"num_read_kb":18804,"num_write":1058,"num_write_kb":5864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"7"}]},{"pgid":"3.18","version":"110'618","reported_seq":3076,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T23:51:31.048512+0000","last_change":"2024-04-23T23:51:31.051535+0000","last_active":"2024-04-23T23:51:31.048512+0000","last_peered":"2024-04-23T23:51:31.048512+0000","last_clean":"2024-04-23T23:51:31.048512+0000","last_became_active":"2024-04-23T23:50:36.142717+0000","last_became_peered":"2024-04-23T23:50:36.142717+0000","last_unstale":"2024-04-23T23:51:31.048512+0000","last_undegraded":"2024-04-23T23:51:31.048512+0000","last_fullsized":"2024-04-23T23:51:31.048512+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:27:39.331634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004592278,"stat_sum":{"num_bytes":19408207,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":3960,"num_read_kb":16768,"num_write":799,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"9"}]},{"pgid":"3.17","version":"107'602","reported_seq":6018,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-23T23:51:29.957196+0000","last_change":"2024-04-23T23:51:27.960222+0000","last_active":"2024-04-23T23:51:29.957196+0000","last_peered":"2024-04-23T23:51:29.957196+0000","last_clean":"2024-04-23T23:51:29.957196+0000","last_became_active":"2024-04-23T23:49:51.313933+0000","last_became_peered":"2024-04-23T23:49:51.313933+0000","last_unstale":"2024-04-23T23:51:29.957196+0000","last_undegraded":"2024-04-23T23:51:29.957196+0000","last_fullsized":"2024-04-23T23:51:29.957196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:59.197424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077631,"stat_sum":{"num_bytes":51816854,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":7308,"num_read_kb":34365,"num_write":701,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.16","version":"107'451","reported_seq":6371,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959729+0000","last_change":"2024-04-23T23:51:27.959694+0000","last_active":"2024-04-23T23:51:28.959729+0000","last_peered":"2024-04-23T23:51:28.959729+0000","last_clean":"2024-04-23T23:51:28.959729+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:51:28.959729+0000","last_undegraded":"2024-04-23T23:51:28.959729+0000","last_fullsized":"2024-04-23T23:51:28.959729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:53:29.011444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644459,"stat_sum":{"num_bytes":41791139,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":7428,"num_read_kb":40464,"num_write":374,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.15","version":"106'336","reported_seq":1991,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955833+0000","last_change":"2024-04-23T23:51:27.956008+0000","last_active":"2024-04-23T23:51:28.955833+0000","last_peered":"2024-04-23T23:51:28.955833+0000","last_clean":"2024-04-23T23:51:28.955833+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:51:28.955833+0000","last_undegraded":"2024-04-23T23:51:28.955833+0000","last_fullsized":"2024-04-23T23:51:28.955833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:12:34.168530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":24266770,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":952,"num_read_kb":13052,"num_write":301,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.14","version":"106'271","reported_seq":4255,"reported_epoch":108,"state":"remapped+peering","last_fresh":"2024-04-23T23:51:27.953071+0000","last_change":"2024-04-23T23:51:28.954423+0000","last_active":"2024-04-23T23:51:27.953071+0000","last_peered":"2024-04-23T23:51:26.037300+0000","last_clean":"2024-04-23T23:51:26.037300+0000","last_became_active":"2024-04-23T23:51:10.332998+0000","last_became_peered":"2024-04-23T23:51:10.332998+0000","last_unstale":"2024-04-23T23:51:27.953071+0000","last_undegraded":"2024-04-23T23:51:27.953071+0000","last_fullsized":"2024-04-23T23:51:27.953071+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00075632900000000003,"stat_sum":{"num_bytes":19272867,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":10,"num_read":3732,"num_read_kb":20797,"num_write":243,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":4028922,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959314+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:51:28.959314+0000","last_peered":"2024-04-23T23:51:28.959314+0000","last_clean":"2024-04-23T23:51:28.959314+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:51:28.959314+0000","last_undegraded":"2024-04-23T23:51:28.959314+0000","last_fullsized":"2024-04-23T23:51:28.959314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'805","reported_seq":2476,"reported_epoch":110,"state":"active+undersized+remapped","last_fresh":"2024-04-23T23:51:30.206523+0000","last_change":"2024-04-23T23:51:30.206757+0000","last_active":"2024-04-23T23:51:30.206523+0000","last_peered":"2024-04-23T23:51:30.206523+0000","last_clean":"2024-04-23T23:51:27.107825+0000","last_became_active":"2024-04-23T23:51:29.982932+0000","last_became_peered":"2024-04-23T23:51:29.982932+0000","last_unstale":"2024-04-23T23:51:30.206523+0000","last_undegraded":"2024-04-23T23:51:30.206523+0000","last_fullsized":"2024-04-23T23:51:29.956876+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048908909999999996,"stat_sum":{"num_bytes":26456923,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":16,"num_read":4203,"num_read_kb":10485,"num_write":1130,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":657077,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","1","4"],"object_location_counts":[{"shards":"0","objects":27}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.963359+0000","last_change":"2024-04-23T23:51:28.962114+0000","last_active":"2024-04-23T23:51:28.963359+0000","last_peered":"2024-04-23T23:51:28.963359+0000","last_clean":"2024-04-23T23:51:28.963359+0000","last_became_active":"2024-04-23T23:51:28.961879+0000","last_became_peered":"2024-04-23T23:51:28.961879+0000","last_unstale":"2024-04-23T23:51:28.963359+0000","last_undegraded":"2024-04-23T23:51:28.963359+0000","last_fullsized":"2024-04-23T23:51:28.963359+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:54.866013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"106'623","reported_seq":4509,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T23:51:31.049081+0000","last_change":"2024-04-23T23:51:31.049409+0000","last_active":"2024-04-23T23:51:31.049081+0000","last_peered":"2024-04-23T23:51:31.049081+0000","last_clean":"2024-04-23T23:51:31.049081+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:51:31.049081+0000","last_undegraded":"2024-04-23T23:51:31.049081+0000","last_fullsized":"2024-04-23T23:51:31.049081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:33:11.961315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343843,"stat_sum":{"num_bytes":9437184,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":646,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":188,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955931+0000","last_change":"2024-04-23T23:49:43.264622+0000","last_active":"2024-04-23T23:51:28.955931+0000","last_peered":"2024-04-23T23:51:28.955931+0000","last_clean":"2024-04-23T23:51:28.955931+0000","last_became_active":"2024-04-23T23:49:43.264206+0000","last_became_peered":"2024-04-23T23:49:43.264206+0000","last_unstale":"2024-04-23T23:51:28.955931+0000","last_undegraded":"2024-04-23T23:51:28.955931+0000","last_fullsized":"2024-04-23T23:51:28.955931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:43.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"107'431","reported_seq":2228,"reported_epoch":108,"state":"unknown","last_fresh":"2024-04-23T23:51:27.956763+0000","last_change":"2024-04-23T23:51:28.395395+0000","last_active":"2024-04-23T23:51:27.956763+0000","last_peered":"2024-04-23T23:51:27.128946+0000","last_clean":"2024-04-23T23:51:27.128946+0000","last_became_active":"2024-04-23T23:49:51.317047+0000","last_became_peered":"2024-04-23T23:49:51.317047+0000","last_unstale":"2024-04-23T23:51:27.956763+0000","last_undegraded":"2024-04-23T23:51:27.956763+0000","last_fullsized":"2024-04-23T23:51:27.956763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020189169999999998,"stat_sum":{"num_bytes":45062995,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":3491,"num_read_kb":9046,"num_write":366,"num_write_kb":5162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"8"}]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-23T23:51:29.957145+0000","last_change":"2024-04-23T23:51:28.960209+0000","last_active":"2024-04-23T23:51:29.957145+0000","last_peered":"2024-04-23T23:51:29.957145+0000","last_clean":"2024-04-23T23:51:29.957145+0000","last_became_active":"2024-04-23T23:51:28.959805+0000","last_became_peered":"2024-04-23T23:51:28.959805+0000","last_unstale":"2024-04-23T23:51:29.957145+0000","last_undegraded":"2024-04-23T23:51:29.957145+0000","last_fullsized":"2024-04-23T23:51:29.957145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:19:54.351275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"107'499","reported_seq":8805,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959949+0000","last_change":"2024-04-23T23:51:27.960530+0000","last_active":"2024-04-23T23:51:28.959949+0000","last_peered":"2024-04-23T23:51:28.959949+0000","last_clean":"2024-04-23T23:51:28.959949+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:51:28.959949+0000","last_undegraded":"2024-04-23T23:51:28.959949+0000","last_fullsized":"2024-04-23T23:51:28.959949+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:44.445137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066127799999999995,"stat_sum":{"num_bytes":29491275,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":9254,"num_read_kb":39295,"num_write":540,"num_write_kb":11263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.3","version":"107'660","reported_seq":4317,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.954210+0000","last_change":"2024-04-23T23:51:27.954125+0000","last_active":"2024-04-23T23:51:28.954210+0000","last_peered":"2024-04-23T23:51:28.954210+0000","last_clean":"2024-04-23T23:51:28.954210+0000","last_became_active":"2024-04-23T23:49:51.317475+0000","last_became_peered":"2024-04-23T23:49:51.317475+0000","last_unstale":"2024-04-23T23:51:28.954210+0000","last_undegraded":"2024-04-23T23:51:28.954210+0000","last_fullsized":"2024-04-23T23:51:28.954210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:29.348829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078122399999999996,"stat_sum":{"num_bytes":3223476,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":12,"num_read":5876,"num_read_kb":22724,"num_write":740,"num_write_kb":8034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"2.2","version":"0'0","reported_seq":193,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.963621+0000","last_change":"2024-04-23T23:51:28.963621+0000","last_active":"2024-04-23T23:51:28.963621+0000","last_peered":"2024-04-23T23:51:28.963621+0000","last_clean":"2024-04-23T23:51:28.963621+0000","last_became_active":"2024-04-23T23:51:28.963362+0000","last_became_peered":"2024-04-23T23:51:28.963362+0000","last_unstale":"2024-04-23T23:51:28.963621+0000","last_undegraded":"2024-04-23T23:51:28.963621+0000","last_fullsized":"2024-04-23T23:51:28.963621+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:57.074807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"106'287","reported_seq":2308,"reported_epoch":108,"state":"unknown","last_fresh":"2024-04-23T23:51:27.957400+0000","last_change":"2024-04-23T23:51:28.395348+0000","last_active":"2024-04-23T23:51:27.957400+0000","last_peered":"2024-04-23T23:51:26.038898+0000","last_clean":"2024-04-23T23:51:26.038898+0000","last_became_active":"2024-04-23T23:49:51.315992+0000","last_became_peered":"2024-04-23T23:49:51.315992+0000","last_unstale":"2024-04-23T23:51:27.957400+0000","last_undegraded":"2024-04-23T23:51:27.957400+0000","last_fullsized":"2024-04-23T23:51:27.957400+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00053349500000000004,"stat_sum":{"num_bytes":14190092,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":2033,"num_read_kb":9958,"num_write":248,"num_write_kb":5983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":192,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-23T23:51:27.956970+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:51:27.956970+0000","last_peered":"2024-04-23T23:51:27.956970+0000","last_clean":"2024-04-23T23:51:27.956970+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:51:27.956970+0000","last_undegraded":"2024-04-23T23:51:27.956970+0000","last_fullsized":"2024-04-23T23:51:27.956970+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'351","reported_seq":3074,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-23T23:51:29.956950+0000","last_change":"2024-04-23T23:51:27.959211+0000","last_active":"2024-04-23T23:51:29.956950+0000","last_peered":"2024-04-23T23:51:29.956950+0000","last_clean":"2024-04-23T23:51:29.956950+0000","last_became_active":"2024-04-23T23:49:51.320719+0000","last_became_peered":"2024-04-23T23:49:51.320719+0000","last_unstale":"2024-04-23T23:51:29.956950+0000","last_undegraded":"2024-04-23T23:51:29.956950+0000","last_fullsized":"2024-04-23T23:51:29.956950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:07.343214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045888599999999998,"stat_sum":{"num_bytes":2621440,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":193,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959742+0000","last_change":"2024-04-23T23:51:28.959742+0000","last_active":"2024-04-23T23:51:28.959742+0000","last_peered":"2024-04-23T23:51:28.959742+0000","last_clean":"2024-04-23T23:51:28.959742+0000","last_became_active":"2024-04-23T23:51:28.959407+0000","last_became_peered":"2024-04-23T23:51:28.959407+0000","last_unstale":"2024-04-23T23:51:28.959742+0000","last_undegraded":"2024-04-23T23:51:28.959742+0000","last_fullsized":"2024-04-23T23:51:28.959742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:13.498370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"106'611","reported_seq":1871,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T23:51:31.048568+0000","last_change":"2024-04-23T23:51:31.048985+0000","last_active":"2024-04-23T23:51:31.048568+0000","last_peered":"2024-04-23T23:51:31.048568+0000","last_clean":"2024-04-23T23:51:31.048568+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:51:31.048568+0000","last_undegraded":"2024-04-23T23:51:31.048568+0000","last_fullsized":"2024-04-23T23:51:31.048568+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:40.621932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019106990000000001,"stat_sum":{"num_bytes":5636157,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":15,"num_read":5753,"num_read_kb":22329,"num_write":786,"num_write_kb":14600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"9"}]},{"pgid":"2.3","version":"0'0","reported_seq":188,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959662+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:51:28.959662+0000","last_peered":"2024-04-23T23:51:28.959662+0000","last_clean":"2024-04-23T23:51:28.959662+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:51:28.959662+0000","last_undegraded":"2024-04-23T23:51:28.959662+0000","last_fullsized":"2024-04-23T23:51:28.959662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":344,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955760+0000","last_change":"2024-04-23T23:49:37.386526+0000","last_active":"2024-04-23T23:51:28.955760+0000","last_peered":"2024-04-23T23:51:28.955760+0000","last_clean":"2024-04-23T23:51:28.955760+0000","last_became_active":"2024-04-23T23:49:37.385426+0000","last_became_peered":"2024-04-23T23:49:37.385426+0000","last_unstale":"2024-04-23T23:51:28.955760+0000","last_undegraded":"2024-04-23T23:51:28.955760+0000","last_fullsized":"2024-04-23T23:51:28.955760+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:12:06.317392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"106'317","reported_seq":1620,"reported_epoch":110,"state":"peering","last_fresh":"2024-04-23T23:51:30.392907+0000","last_change":"2024-04-23T23:51:31.047611+0000","last_active":"2024-04-23T23:51:31.047611+0000","last_peered":"2024-04-23T23:51:30.392907+0000","last_clean":"2024-04-23T23:51:27.041008+0000","last_became_active":"2024-04-23T23:51:30.057014+0000","last_became_peered":"2024-04-23T23:51:30.057014+0000","last_unstale":"2024-04-23T23:51:30.392907+0000","last_undegraded":"2024-04-23T23:51:30.392907+0000","last_fullsized":"2024-04-23T23:51:29.957448+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021399499999999998,"stat_sum":{"num_bytes":13677979,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":12,"num_read":4222,"num_read_kb":8442,"num_write":283,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"7"}]},{"pgid":"3.9","version":"108'476","reported_seq":2580,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.954259+0000","last_change":"2024-04-23T23:51:27.953430+0000","last_active":"2024-04-23T23:51:28.954259+0000","last_peered":"2024-04-23T23:51:28.954259+0000","last_clean":"2024-04-23T23:51:28.954259+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:51:28.954259+0000","last_undegraded":"2024-04-23T23:51:28.954259+0000","last_fullsized":"2024-04-23T23:51:28.954259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:53.856943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000528624,"stat_sum":{"num_bytes":4716399,"num_objects":24,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":3970,"num_read_kb":12269,"num_write":397,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.a","version":"107'424","reported_seq":3349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959278+0000","last_change":"2024-04-23T23:51:27.961920+0000","last_active":"2024-04-23T23:51:28.959278+0000","last_peered":"2024-04-23T23:51:28.959278+0000","last_clean":"2024-04-23T23:51:28.959278+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:51:28.959278+0000","last_undegraded":"2024-04-23T23:51:28.959278+0000","last_fullsized":"2024-04-23T23:51:28.959278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:50:01.785817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044401899999999998,"stat_sum":{"num_bytes":45934117,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":18,"num_read":2777,"num_read_kb":16922,"num_write":398,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.b","version":"110'411","reported_seq":2440,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T23:51:31.049500+0000","last_change":"2024-04-23T23:51:31.049784+0000","last_active":"2024-04-23T23:51:31.049500+0000","last_peered":"2024-04-23T23:51:31.049500+0000","last_clean":"2024-04-23T23:51:31.049500+0000","last_became_active":"2024-04-23T23:50:32.954861+0000","last_became_peered":"2024-04-23T23:50:32.954861+0000","last_unstale":"2024-04-23T23:51:31.049500+0000","last_undegraded":"2024-04-23T23:51:31.049500+0000","last_fullsized":"2024-04-23T23:51:31.049500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:48.885715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025131960000000001,"stat_sum":{"num_bytes":2228326,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":2779,"num_read_kb":21636,"num_write":427,"num_write_kb":8262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":3287017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"9"}]},{"pgid":"3.c","version":"106'424","reported_seq":3913,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.960624+0000","last_change":"2024-04-23T23:51:27.959836+0000","last_active":"2024-04-23T23:51:28.960624+0000","last_peered":"2024-04-23T23:51:28.960624+0000","last_clean":"2024-04-23T23:51:28.960624+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:51:28.960624+0000","last_undegraded":"2024-04-23T23:51:28.960624+0000","last_fullsized":"2024-04-23T23:51:28.960624+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:42.081019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063734299999999996,"stat_sum":{"num_bytes":29607053,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":3937,"num_read_kb":29122,"num_write":363,"num_write_kb":15932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.d","version":"107'353","reported_seq":3350,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955872+0000","last_change":"2024-04-23T23:51:27.956358+0000","last_active":"2024-04-23T23:51:28.955872+0000","last_peered":"2024-04-23T23:51:28.955872+0000","last_clean":"2024-04-23T23:51:28.955872+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:51:28.955872+0000","last_undegraded":"2024-04-23T23:51:28.955872+0000","last_fullsized":"2024-04-23T23:51:28.955872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:25.333691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078019299999999999,"stat_sum":{"num_bytes":46281470,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":3559,"num_read_kb":24871,"num_write":320,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.e","version":"107'546","reported_seq":3425,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955782+0000","last_change":"2024-04-23T23:51:27.956062+0000","last_active":"2024-04-23T23:51:28.955782+0000","last_peered":"2024-04-23T23:51:28.955782+0000","last_clean":"2024-04-23T23:51:28.955782+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:51:28.955782+0000","last_undegraded":"2024-04-23T23:51:28.955782+0000","last_fullsized":"2024-04-23T23:51:28.955782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:20.920285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098174799999999995,"stat_sum":{"num_bytes":5873836,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":4344,"num_read_kb":16252,"num_write":680,"num_write_kb":2766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.f","version":"106'851","reported_seq":3884,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955795+0000","last_change":"2024-04-23T23:51:27.956274+0000","last_active":"2024-04-23T23:51:28.955795+0000","last_peered":"2024-04-23T23:51:28.955795+0000","last_clean":"2024-04-23T23:51:28.955795+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:51:28.955795+0000","last_undegraded":"2024-04-23T23:51:28.955795+0000","last_fullsized":"2024-04-23T23:51:28.955795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:07.384262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010727340000000001,"stat_sum":{"num_bytes":32039796,"num_objects":22,"num_object_clones":1,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":5720,"num_read_kb":22932,"num_write":1217,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.10","version":"106'316","reported_seq":5422,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.959352+0000","last_change":"2024-04-23T23:51:27.962624+0000","last_active":"2024-04-23T23:51:28.959352+0000","last_peered":"2024-04-23T23:51:28.959352+0000","last_clean":"2024-04-23T23:51:28.959352+0000","last_became_active":"2024-04-23T23:49:51.316266+0000","last_became_peered":"2024-04-23T23:49:51.316266+0000","last_unstale":"2024-04-23T23:51:28.959352+0000","last_undegraded":"2024-04-23T23:51:28.959352+0000","last_fullsized":"2024-04-23T23:51:28.959352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:32.004094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042465299999999999,"stat_sum":{"num_bytes":5044970,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":16,"num_read":5432,"num_read_kb":16759,"num_write":234,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.11","version":"106'408","reported_seq":4059,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955645+0000","last_change":"2024-04-23T23:51:27.955548+0000","last_active":"2024-04-23T23:51:28.955645+0000","last_peered":"2024-04-23T23:51:28.955645+0000","last_clean":"2024-04-23T23:51:28.955645+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:51:28.955645+0000","last_undegraded":"2024-04-23T23:51:28.955645+0000","last_fullsized":"2024-04-23T23:51:28.955645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:19.318413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307199999999996,"stat_sum":{"num_bytes":2768896,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":364,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.12","version":"110'414","reported_seq":1945,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-23T23:51:29.956972+0000","last_change":"2024-04-23T23:51:27.958092+0000","last_active":"2024-04-23T23:51:29.956972+0000","last_peered":"2024-04-23T23:51:29.956972+0000","last_clean":"2024-04-23T23:51:29.956972+0000","last_became_active":"2024-04-23T23:49:51.318462+0000","last_became_peered":"2024-04-23T23:49:51.318462+0000","last_unstale":"2024-04-23T23:51:29.956972+0000","last_undegraded":"2024-04-23T23:51:29.956972+0000","last_fullsized":"2024-04-23T23:51:29.956972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:49.594298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001445984,"stat_sum":{"num_bytes":26698384,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1920,"num_read_kb":16622,"num_write":395,"num_write_kb":10455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"f","length":"8"}]},{"pgid":"3.13","version":"106'454","reported_seq":2827,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-23T23:51:28.955687+0000","last_change":"2024-04-23T23:51:27.955755+0000","last_active":"2024-04-23T23:51:28.955687+0000","last_peered":"2024-04-23T23:51:28.955687+0000","last_clean":"2024-04-23T23:51:28.955687+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:51:28.955687+0000","last_undegraded":"2024-04-23T23:51:28.955687+0000","last_fullsized":"2024-04-23T23:51:28.955687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:48.816204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079356299999999995,"stat_sum":{"num_bytes":29345660,"num_objects":22,"num_object_clones":1,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":3908,"num_read_kb":14853,"num_write":386,"num_write_kb":16095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"f","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":601162611,"num_objects":636,"num_object_clones":43,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":636,"num_whiteouts":421,"num_read":153402,"num_read_kb":611596,"num_write":17052,"num_write_kb":302957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":681,"num_bytes_recovered":19769765,"num_keys_recovered":138,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117198848,"data_stored":146941507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":15831,"ondisk_log_size":15831,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168589,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66356,"kb_used_data":25312,"kb_used_omap":10,"kb_used_meta":41013,"kb_avail":94305484,"statfs":{"total":96636764160,"available":96568815616,"internally_reserved":0,"allocated":25919488,"data_stored":32192853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":41997578},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509468,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63140,"kb_used_data":13904,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308700,"statfs":{"total":96636764160,"available":96572108800,"internally_reserved":0,"allocated":14237696,"data_stored":14641489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57088,"kb_used_data":25904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314752,"statfs":{"total":96636764160,"available":96578306048,"internally_reserved":0,"allocated":26525696,"data_stored":33439330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1937408,"data_stored":1186577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65264,"kb_used_data":38240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94306576,"statfs":{"total":96636764160,"available":96569933824,"internally_reserved":0,"allocated":39157760,"data_stored":50765479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59492,"kb_used_data":10264,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312348,"statfs":{"total":96636764160,"available":96575844352,"internally_reserved":0,"allocated":10510336,"data_stored":11173619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37532,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334308,"statfs":{"total":96636764160,"available":96598331392,"internally_reserved":0,"allocated":2244608,"data_stored":1886147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12099584,"data_stored":13213737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":7309124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":978944,"data_stored":1288928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24616960,"data_stored":31576671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9232384,"data_stored":10569793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37879808,"data_stored":50161653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":659456,"data_stored":582751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24641536,"data_stored":32238850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:51:33.767 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [5, 3] 2024-04-23T23:51:33.767 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '5', '3'] 2024-04-23T23:51:33.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 5 3 2024-04-23T23:51:33.773 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:33.787+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:33.874 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:33.888+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:33.911 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:33.912 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:33.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:34.005 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:34.019+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:34.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:51:34.075 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:34.075 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:34.076 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:34.076 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:34.076 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:34.076 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:34.089+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:34.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:34.177 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:34.191+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:34.240 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.739745377000006, 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:34.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.739279389999993, 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:34.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:34.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.738772840999999, 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:34.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:34.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.738476933000001, 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:34.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.73828992, 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:34.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:34.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:34.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.738088250000004, 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:34.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:34.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.737947988000002, 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:34.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.737702252999995, 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:34.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.737424727999993, 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:34.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.737182156000003, 2024-04-23T23:51:34.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.737003791999996, 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:34.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.736721200999995, 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:34.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.736528716999999, 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:34.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:34.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.736100776000001, 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:34.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:34.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.735948442999998, 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:34.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:34.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.735845359999999, 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:34.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:34.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.735528414000001, 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:34.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.735286986999995, 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:34.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.735062439999993, 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.734807543000002, 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.328 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:34.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:34.330 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:34.343+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:34.431 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:34.445+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:34.473 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:34.473 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:34.473 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:34.473 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:34.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:34.532 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:34.546+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:34.583 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:34.583 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:34.583 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:34.583 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:34.583 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:34.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:34.633 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:34.646+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:34.734 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:34.748+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:34.735 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 93.289035552000001, 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.479529057000001, 2024-04-23T23:51:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.429734170000003, 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.383109101999999, 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.380338727999998, 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.284952898999997, 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.282534816999998, 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.563787546999997, 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.388204096000003, 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1451 3.9 e95/22 3:9566a16b:::rbd_data.117911508d31.0000000000000023:1c v 95'434, mlcod=91'429)", 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.789953031, 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043290560000000004, 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.68e-07 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958651+0000", 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958664+0000", 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3604000000000001e-05 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958671+0000", 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958719+0000", 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7568000000000003e-05 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958742+0000", 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3532000000000001e-05 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962919+0000", 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041768550000000002 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962974+0000", 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4691999999999997e-05 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1375 3.0 e96/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:1b v 96'269, mlcod=95'264)", 2024-04-23T23:51:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.788333657999999, 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054529929999999997, 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960265+0000", 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.61e-07 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960271+0000", 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2750000000000001e-06 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960282+0000", 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056000000000001e-05 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960289+0000", 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7660000000000001e-06 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960552+0000", 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026309199999999998 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960571+0000", 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8980000000000001e-05 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965676+0000", 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051052739999999999 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965717+0000", 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0788999999999999e-05 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1482 3.9 e96/22 3:90d97e73:::rbd_data.11798ae5b3f1.0000000000000008:1b v 96'436, mlcod=95'432)", 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.784235109000001, 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045767280000000004, 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964368+0000", 2024-04-23T23:51:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964386+0000", 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7876e-05 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964392+0000", 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964440+0000", 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7321999999999999e-05 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964459+0000", 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9313e-05 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968889+0000", 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004429962 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968939+0000", 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0501000000000003e-05 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1378 3.0 e96/22 3:03a7baf4:::rbd_data.11798ae5b3f1.000000000000024d:1b v 96'271, mlcod=96'267)", 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.781872064000002, 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007336375, 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966731+0000", 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.011e-06 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966740+0000", 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9579999999999996e-06 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966745+0000", 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966788+0000", 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985000000000003e-05 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966803+0000", 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2024-04-23T23:51:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974010+0000", 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072070320000000004 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974062+0000", 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1805999999999998e-05 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1483 3.9 e96/22 3:94dae484:::rbd_data.11798ae5b3f1.000000000000006c:1b v 96'438, mlcod=95'432)", 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.780942035999999, 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089730109999999995, 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967662+0000", 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:51:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967675+0000", 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2763e-05 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967683+0000", 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5530000000000004e-06 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967734+0000", 2024-04-23T23:51:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967756+0000", 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1906999999999999e-05 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976580+0000", 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088237190000000007 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976629+0000", 2024-04-23T23:51:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8948e-05 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1486 3.9 e96/22 3:96d7965c:::rbd_data.11798ae5b3f1.00000000000003a1:1b v 96'442, mlcod=96'436)", 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.768498021999999, 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092063349999999995, 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980107+0000", 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4180000000000002e-06 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980117+0000", 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0565e-05 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980124+0000", 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980185+0000", 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980206+0000", 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1472e-05 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989259+0000", 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090529410000000001 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989306+0000", 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950000000000003e-05 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1383 3.4 e96/22 3:212acb2d:::rbd_data.11798ae5b3f1.0000000000000184:1b v 96'397, mlcod=96'393)", 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.762806681000001, 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043511569999999996, 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985792+0000", 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.062e-06 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985801+0000", 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985819+0000", 2024-04-23T23:51:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-05 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985827+0000", 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8599999999999999e-06 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985877+0000", 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790999999999998e-05 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985900+0000", 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2418000000000001e-05 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990100+0000", 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042000819999999999 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990142+0000", 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2469999999999998e-05 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1384 3.4 e96/22 3:25b57ffa:::rbd_data.11798ae5b3f1.000000000000006e:1b v 96'399, mlcod=96'393)", 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.762148267000001, 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045474490000000003, 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986450+0000", 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986457+0000", 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9809999999999997e-06 2024-04-23T23:51:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986473+0000", 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6016e-05 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986481+0000", 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986530+0000", 2024-04-23T23:51:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8773000000000002e-05 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986550+0000", 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0347999999999999e-05 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990945+0000", 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043951370000000003 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990997+0000", 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1674e-05 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119499 3.0 3:01c0afba:::rbd_data.1179c595b6df.0000000000000004:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.7016136550000001, 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094379389999999994, 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119499 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046985+0000", 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4079999999999999e-06 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046991+0000", 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046999+0000", 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1990000000000006e-06 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047007+0000", 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3820000000000004e-06 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047532+0000", 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000524472 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047547+0000", 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4793e-05 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054754+0000", 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072074249999999999 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054959+0000", 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000205136 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055054+0000", 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4697999999999994e-05 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055940+0000", 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088648299999999998 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056318+0000", 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037777699999999998 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056354+0000", 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6106999999999998e-05 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056422+0000", 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7639999999999996e-05 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119508 3.0 3:05062586:::rbd_data.1179c595b6df.000000000000000d:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6994201589999998, 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007607613, 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119508 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049179+0000", 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.125e-06 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049184+0000", 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6579999999999999e-06 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049195+0000", 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0736e-05 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049203+0000", 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049260+0000", 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6839999999999998e-05 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049270+0000", 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.893e-06 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055559+0000", 2024-04-23T23:51:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062896660000000002 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055647+0000", 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7496000000000004e-05 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055726+0000", 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8917000000000004e-05 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056225+0000", 2024-04-23T23:51:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049874700000000001 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056601+0000", 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037684200000000001 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056663+0000", 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1779000000000006e-05 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056785+0000", 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122019 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119713 3.0 3:0770b89d:::rbd_header.1179129e5779:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6881476119999999, 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050942269999999998, 2024-04-23T23:51:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119713 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060452+0000", 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3099999999999999e-06 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060462+0000", 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6800000000000005e-06 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060485+0000", 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2864999999999999e-05 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060496+0000", 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1287999999999999e-05 2024-04-23T23:51:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060567+0000", 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0324e-05 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060573+0000", 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8719999999999996e-06 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063243+0000", 2024-04-23T23:51:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002669158 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063352+0000", 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109819 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063761+0000", 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040853800000000001 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.064740+0000", 2024-04-23T23:51:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097912600000000008 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065441+0000", 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070055300000000003 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065500+0000", 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9769000000000003e-05 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065544+0000", 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3924999999999997e-05 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:34.831 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:34.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:34.835 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:34.849+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:34.936 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:34.950+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:34.947 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:34.947 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:34.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:35.037 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:35.051+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:set 3.12 pg_upmap mapping to [5,3] 2024-04-23T23:51:35.050 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:51:35.102 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:35.103 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:35.103 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:35.103 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:35.103 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:35.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:35.139 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:35.153+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:35.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:35.215 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:35.215 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.797527224999996, 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.216 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:35.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.797517345000003, 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:35.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:35.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:35.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.797306997999996, 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:35.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:35.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.229 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.795475659000004, 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.793406028000007, 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:35.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:35.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281315792000001, 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:35.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:35.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281290135999996, 2024-04-23T23:51:35.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:35.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281222646000003, 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.250 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:35.254 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281177584999995, 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:35.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:35.258 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281141085000002, 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.263 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281124125999995, 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:35.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:35.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.281045442000007, 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:35.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.280978408999999, 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:35.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:35.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:35.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.280844807999998, 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:35.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.280426128000002, 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:35.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:35.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.280275485999994, 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:35.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.280150104000001, 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:35.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.279934587, 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:35.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:35.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.279697252999995, 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:35.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.279457358000002, 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:35.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:35.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:35.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.308 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:35.308 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:35.308 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:35.309 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:35.309 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:35.310 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:35.310 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:35.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:35.311 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:35.323+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:35.311 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:35.312 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:35.323 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:35.323 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:35.340 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:35.357 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:35.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:35.392 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:35.393 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:35.409 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:35.423+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:35.452 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:35.452 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:35.461 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:35.510 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:35.524+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:35.563 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:35.563 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:35.572 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:35.616 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:35.630+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:35.674 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:35.674 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:35.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:35.712 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:35.726+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:35.814 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:35.828+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:35.826 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:35.826 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:35.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:35.914 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:35.928+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:35.937 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:35.937 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:35.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:36.030 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:36.044+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:36.093 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.093 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:36.143 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:36.157+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:36.218 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:36.232+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:36.245 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.245 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:36.319 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:36.333+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:36.396 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.397 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:36.449 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:36.463+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:36.508 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.508 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:36.558 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:36.572+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:36.622 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:36.636+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:36.624 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.624 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:36.723 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:36.737+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:36.735 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.735 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:36.824 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:36.838+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:36.888 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:36.888 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:36.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:36.925 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:36.939+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:36.999 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:37.000 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:37.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:37.053 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:37.067+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:37.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:51:37.126 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:37.140+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:37.129 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:37.129 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:37.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:37.227 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:37.241+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:37.266 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:37.267 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:37.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:37.328 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:37.342+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:37.379 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:37.379 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:37.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:37.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:37.432 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:37.446+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:37.492 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:37.492 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:37.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:37.529 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:37.543+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:37.552 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:37.585 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:37.602 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:37.602 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:37.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.123315387999995, 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.122849400999996, 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.122342852000003, 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.122046944000004, 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.121859931000003, 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.121658260999993, 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.121517999000005, 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.121272263999998, 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.120994738999997, 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.120752167000006, 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.120573802999999, 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.120291211999998, 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.120098728000002, 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.119670787000004, 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.119518454000001, 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.119415371000002, 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.119098425000004, 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.118856997999998, 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.118632450999996, 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.118377554000006, 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:37.716 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:37.717 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:37.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:37.719 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:37.719 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:37.720 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:37.720 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:37.720 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:37.721 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:37.722 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:37.735+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:37.822 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:37.836+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:37.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:37.923 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:37.937+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:37.937 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:51:38.024 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:38.038+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:38.030 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:38.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:38.031 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:38.031 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:38.031 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:38.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:38.125 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:38.139+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:38.147 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.701393541000002, 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.891887046000001, 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.842092159000003, 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.795467090999999, 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.792696716999998, 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.697310887999997, 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.694892805999999, 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.976145535999997, 2024-04-23T23:51:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.800562085000003, 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1451 3.9 e95/22 3:9566a16b:::rbd_data.117911508d31.0000000000000023:1c v 95'434, mlcod=91'429)", 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.20231102, 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043290560000000004, 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.68e-07 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958651+0000", 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958664+0000", 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3604000000000001e-05 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958671+0000", 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-23T23:51:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958719+0000", 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7568000000000003e-05 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958742+0000", 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3532000000000001e-05 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962919+0000", 2024-04-23T23:51:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041768550000000002 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962974+0000", 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4691999999999997e-05 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1375 3.0 e96/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:1b v 96'269, mlcod=95'264)", 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.200691646999999, 2024-04-23T23:51:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054529929999999997, 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960265+0000", 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.61e-07 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960271+0000", 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2750000000000001e-06 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960282+0000", 2024-04-23T23:51:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056000000000001e-05 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960289+0000", 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7660000000000001e-06 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960552+0000", 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026309199999999998 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960571+0000", 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8980000000000001e-05 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965676+0000", 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051052739999999999 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965717+0000", 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0788999999999999e-05 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1482 3.9 e96/22 3:90d97e73:::rbd_data.11798ae5b3f1.0000000000000008:1b v 96'436, mlcod=95'432)", 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.196593098000001, 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045767280000000004, 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964368+0000", 2024-04-23T23:51:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964386+0000", 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7876e-05 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964392+0000", 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964440+0000", 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7321999999999999e-05 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964459+0000", 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9313e-05 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968889+0000", 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004429962 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968939+0000", 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0501000000000003e-05 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1378 3.0 e96/22 3:03a7baf4:::rbd_data.11798ae5b3f1.000000000000024d:1b v 96'271, mlcod=96'267)", 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.194230052999998, 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007336375, 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-23T23:51:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966731+0000", 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.011e-06 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966740+0000", 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9579999999999996e-06 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966745+0000", 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966788+0000", 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985000000000003e-05 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966803+0000", 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974010+0000", 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072070320000000004 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974062+0000", 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1805999999999998e-05 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1483 3.9 e96/22 3:94dae484:::rbd_data.11798ae5b3f1.000000000000006c:1b v 96'438, mlcod=95'432)", 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.193300024999999, 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089730109999999995, 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967662+0000", 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967675+0000", 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2763e-05 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967683+0000", 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5530000000000004e-06 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967734+0000", 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967756+0000", 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1906999999999999e-05 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976580+0000", 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088237190000000007 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976629+0000", 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8948e-05 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1486 3.9 e96/22 3:96d7965c:::rbd_data.11798ae5b3f1.00000000000003a1:1b v 96'442, mlcod=96'436)", 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.180856010999999, 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092063349999999995, 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980107+0000", 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4180000000000002e-06 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980117+0000", 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0565e-05 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980124+0000", 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980185+0000", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980206+0000", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1472e-05 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989259+0000", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090529410000000001 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989306+0000", 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950000000000003e-05 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1383 3.4 e96/22 3:212acb2d:::rbd_data.11798ae5b3f1.0000000000000184:1b v 96'397, mlcod=96'393)", 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.175164670000001, 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043511569999999996, 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985792+0000", 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.062e-06 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985801+0000", 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985819+0000", 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-05 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985827+0000", 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8599999999999999e-06 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985877+0000", 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790999999999998e-05 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985900+0000", 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2418000000000001e-05 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990100+0000", 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042000819999999999 2024-04-23T23:51:38.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990142+0000", 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2469999999999998e-05 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1384 3.4 e96/22 3:25b57ffa:::rbd_data.11798ae5b3f1.000000000000006e:1b v 96'399, mlcod=96'393)", 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.174506256000001, 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045474490000000003, 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986450+0000", 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986457+0000", 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9809999999999997e-06 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986473+0000", 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6016e-05 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986481+0000", 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986530+0000", 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8773000000000002e-05 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986550+0000", 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0347999999999999e-05 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990945+0000", 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043951370000000003 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990997+0000", 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1674e-05 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119499 3.0 3:01c0afba:::rbd_data.1179c595b6df.0000000000000004:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.1139716439999994, 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094379389999999994, 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119499 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046985+0000", 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4079999999999999e-06 2024-04-23T23:51:38.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046991+0000", 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046999+0000", 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1990000000000006e-06 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047007+0000", 2024-04-23T23:51:38.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3820000000000004e-06 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047532+0000", 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000524472 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047547+0000", 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4793e-05 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054754+0000", 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072074249999999999 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054959+0000", 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000205136 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055054+0000", 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4697999999999994e-05 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055940+0000", 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088648299999999998 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056318+0000", 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037777699999999998 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056354+0000", 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6106999999999998e-05 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056422+0000", 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7639999999999996e-05 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119508 3.0 3:05062586:::rbd_data.1179c595b6df.000000000000000d:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.1117781480000009, 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007607613, 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119508 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049179+0000", 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.125e-06 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049184+0000", 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6579999999999999e-06 2024-04-23T23:51:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049195+0000", 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0736e-05 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049203+0000", 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049260+0000", 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6839999999999998e-05 2024-04-23T23:51:38.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049270+0000", 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.893e-06 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055559+0000", 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062896660000000002 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055647+0000", 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7496000000000004e-05 2024-04-23T23:51:38.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055726+0000", 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8917000000000004e-05 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056225+0000", 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049874700000000001 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056601+0000", 2024-04-23T23:51:38.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037684200000000001 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056663+0000", 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1779000000000006e-05 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056785+0000", 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122019 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119713 3.0 3:0770b89d:::rbd_header.1179129e5779:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1005056010000001, 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050942269999999998, 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119713 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060452+0000", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3099999999999999e-06 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:38.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060462+0000", 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6800000000000005e-06 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060485+0000", 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2864999999999999e-05 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060496+0000", 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1287999999999999e-05 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060567+0000", 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0324e-05 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060573+0000", 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8719999999999996e-06 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063243+0000", 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002669158 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063352+0000", 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109819 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063761+0000", 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040853800000000001 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.064740+0000", 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097912600000000008 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065441+0000", 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070055300000000003 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065500+0000", 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9769000000000003e-05 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065544+0000", 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3924999999999997e-05 2024-04-23T23:51:38.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.241 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:38.241 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:38.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:38.243 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:38.256+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:38.343 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:38.357+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:38.347 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:38.347 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:38.348 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:38.348 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:38.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:38.445 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:38.458+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:38.497 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:38.498 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:38.498 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:38.498 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:38.498 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:38.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:38.545 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:38.559+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.192287077000003, 2024-04-23T23:51:38.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:38.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.192277196999996, 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:38.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.192066850000003, 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:38.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.190235510999997, 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:38.623 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:38.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.18816588, 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.628 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:38.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.676075643999994, 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:38.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.676049988000003, 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:38.636 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:38.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:38.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675982497999996, 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.644 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675937437000002, 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:38.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675900936999994, 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.649 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:38.653 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675883978000002, 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:38.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:38.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675805294, 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:38.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:38.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:38.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675738261000006, 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675604660000005, 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.666 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:38.670 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675185979999995, 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:38.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:38.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.675035338000001, 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.675 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:38.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:38.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.674909955999993, 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.674694439000007, 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:38.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:38.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.674457105000002, 2024-04-23T23:51:38.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:38.691 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.674217209999995, 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:38.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:38.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:38.695 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:38.696 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:38.697 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:38.698 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:38.711+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:38.798 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:38.812+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:38.800 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:38.801 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:38.801 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:38.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:38.899 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:38.913+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:38.962 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:38.962 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:38.962 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:38.962 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:38.962 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:38.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:39.013 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:39.027+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:39.102 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:39.115+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:39.123 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:39.123 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:39.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.866156713999999, 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.124 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:39.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.866135834000005, 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.860278047999998, 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.860263801000002, 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.858229867999995, 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.858217925999995, 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:39.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.856240791999994, 2024-04-23T23:51:39.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:39.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:39.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.856212708000001, 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:39.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:39.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:39.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:39.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.856200173000005, 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.854259084999995, 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.854247739000002, 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.167 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:39.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:39.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.854236885000006, 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.171 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:39.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:39.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:39.175 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.854225593999999, 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:39.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.852186821000004, 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.180 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:39.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:39.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.852170838000006, 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:39.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:39.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.852150218999995, 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:39.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.852130220999996, 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.852111941000004, 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.197 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:39.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.851109742999995, 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.201 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:39.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:39.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.851098243999999, 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.205 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.209 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:39.211 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:39.211 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:39.224+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:39.311 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:39.325+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:39.314 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:39.314 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:39.314 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:39.314 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:39.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:39.412 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:39.426+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:39.428 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:39.429 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:39.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:39.514 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:39.528+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.036024873000002, 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:39.542 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:39.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:39.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.546 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.035991824999996, 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.035969326, 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.551 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:39.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:39.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:39.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.034099409000007, 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:39.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.034070932999995, 2024-04-23T23:51:39.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:39.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.034048489, 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:39.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.032092761000001, 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.032073634, 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.572 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.032047485999996, 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.576 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:39.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:39.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.032026794000004, 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:39.580 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:39.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.032005068000004, 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:39.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:39.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.030108178000006, 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:39.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.030083731000005, 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:39.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:39.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:39.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.030066637000004, 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.597 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.028089256000001, 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:39.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.028070377999995, 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:39.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.026089338000006, 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.610 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:39.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.026049536000002, 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:39.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:39.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.026027511999999, 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.622 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 74.026005971000004, 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:39.623 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:39.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:39.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:39.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:39.627 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:39.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:39.629 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:39.642+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:39.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:39.729 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:39.743+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:39.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:39.830 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:39.844+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:39.851 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:39.852 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:39.868 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:39.869 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:39.885 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:39.901 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:39.920 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:39.920 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:39.926 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:39.926 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:39.926 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:39.927 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:39.927 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:39.931 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:39.945+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:39.938 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:39.939 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:39.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-23T23:51:39.959 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:39.981 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:40.031 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:40.045+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:40.066 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:40.066 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:40.066 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 2, 3, 4, 5, 6, 7] 2024-04-23T23:51:40.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-04-23T23:51:40.085 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.086 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:40.136 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:40.150+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:40.219 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.219 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:40.273 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:40.287+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:40.333 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.333 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.334 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:40.348+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:40.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:40.377 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-04-23T23:51:40.390+0000 7f856ad67640 -1 mon.a@0(leader).osd e120 definitely_dead 0 2024-04-23T23:51:40.435 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:40.449+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:40.478 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:40.536 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:40.550+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:40.591 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.592 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:40.637 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:40.651+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:40.738 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:40.752+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:40.741 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.742 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:40.839 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:40.852+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:40.853 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:40.853 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:40.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:40.941 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-04-23T23:51:40.942 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:40.955+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.0. 2024-04-23T23:51:41.013 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.014 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:41.066 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:41.080+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:41.126 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.126 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:41.176 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:41.190+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:41.242 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:41.256+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:41.277 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.277 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:41.343 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:41.357+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:41.429 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.429 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:41.444 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:41.458+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:41.540 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.540 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.545 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:41.559+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:41.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:41.646 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:41.660+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:41.650 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.650 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:41.747 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:41.761+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:41.774 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.774 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:41.848 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:41.862+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:41.885 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:41.886 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:41.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:41.950 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:41.963+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:41.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:42.000 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:42.001 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:42.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:42.050 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:42.064+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:42.112 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:42.113 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:42.116 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:42.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:42.133 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:42.133 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:42.149 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:42.165 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:42.179+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:42.166 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:42.183 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:42.183 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:42.200 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:42.200 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:42.217 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:42.218 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:42.224 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:42.224 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:42.224 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:42.224 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:42.224 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:42.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:42.234 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:42.235 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:42.274 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:42.288+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:42.353 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:42.367+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.12093591, 2024-04-23T23:51:42.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:42.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:42.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.120915030000006, 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:42.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:42.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.115057243999999, 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:42.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:42.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.115042997000003, 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:42.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:42.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.113009063999996, 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:42.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:42.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.112997121999996, 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:42.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:42.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.111019987999995, 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:42.404 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.110991904000002, 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:42.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:42.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.110979369000006, 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.109038280999997, 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:42.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:42.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.109026935000003, 2024-04-23T23:51:42.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:42.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:42.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.109016080999993, 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.430 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.10900479, 2024-04-23T23:51:42.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:42.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:42.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.106966017000005, 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.437 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:42.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:42.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.106950033999993, 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.445 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.106929414999996, 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.449 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.106909416999997, 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.106891137000005, 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.458 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.105888938999996, 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.462 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.10587744, 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.463 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:42.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:42.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:42.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:42.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.467 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.467 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.467 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:42.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:42.469 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:42.483+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:42.569 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:42.583+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:42.613 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:42.613 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:42.613 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:42.613 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:42.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:42.705 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:42.719+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:42.764 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:42.764 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:42.764 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:42.764 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:42.764 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:42.771 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:42.785+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:42.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:42.872 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:42.886+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:42.916 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:42.916 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:42.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.410532388000007, 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:42.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.410499340000001, 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.921 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.925 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.410476841000005, 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:42.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.408606923999997, 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:42.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:42.934 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.408578448, 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:42.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:42.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.408556004000005, 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:42.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:42.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.406600276000006, 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:42.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.406581149000004, 2024-04-23T23:51:42.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:42.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.951 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.406555001000001, 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:42.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:42.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.406534308999994, 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.956 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:42.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:42.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.406512582999994, 2024-04-23T23:51:42.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:42.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:42.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.964 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.404615692999997, 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:42.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:42.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:42.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.404591245999995, 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:42.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.404574151999995, 2024-04-23T23:51:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:42.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.402596771000006, 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:42.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:42.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.402577893, 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.400596852999996, 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:42.990 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.400557051000007, 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:42.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:42.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.400535027000004, 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.400513485999994, 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:43.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.003 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:43.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:51:43.007 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:43.020+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:43.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:43.107 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:43.121+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:43.149 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:43.149 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:43.149 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:43.149 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:43.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:43.243 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:43.257+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:43.309 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:43.323+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:43.323 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:43.324 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:43.324 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:43.324 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:43.324 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:43.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:43.413 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:43.424+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:43.476 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:43.476 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:43.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 126.288841711, 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:43.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:43.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.940105043000003, 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.487 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.940088590000002, 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:43.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:43.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.940061849000003, 2024-04-23T23:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:43.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.496 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.940043054, 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.940025469999995, 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.502 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:43.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.928329966000007, 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:43.506 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.928154278999997, 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:43.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:43.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:43.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.928002759999998, 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:43.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:43.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.927720133999998, 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.525 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.927574269000004, 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:43.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:43.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:43.530 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.926624584999999, 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:43.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.534 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.926601691000002, 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.535 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.538 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.910586316000007, 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.542 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.910576681999999, 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.895544321000003, 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.547 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:43.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.895522997, 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:43.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:43.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.895500171999998, 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:43.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:43.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.895477493000001, 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:43.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.893175769999999, 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:43.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:43.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:43.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:43.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:43.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:43.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:43.570 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:43.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:43.572 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:43.585+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:43.672 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:43.687+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:43.675 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:43.676 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:43.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:43.773 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:43.787+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:43.786 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:43.786 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:43.787 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:43.787 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:43.787 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:43.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:43.879 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:43.893+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:43.939 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:43.939 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 71.837871507000003, 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 71.835277176000005, 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:43.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723621737000002, 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:43.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:43.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:43.955 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723603296999997, 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:43.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723584191, 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:43.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723565682, 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:43.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723462632999997, 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723443013999997, 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723337538999999, 2024-04-23T23:51:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723312354000001, 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723278808999993, 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723255046000006, 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723234004999995, 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723165769999994, 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723141302000002, 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723115604, 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723093837999997, 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.723073372000002, 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.722995513000001, 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.722965861999995, 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:44.032 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:44.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:51:44.034 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:44.047+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:44.133 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:44.147+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:44.175 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:44.175 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:44.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:44.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:44.268 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:44.282+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:44.328 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:44.329 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:44.336 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:44.350+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:44.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:44.388 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:44.407 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:44.407 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:44.423 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:44.437 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:44.451+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:44.441 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:44.441 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:44.458 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:44.476 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:44.476 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:44.484 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:44.484 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:44.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:44.494 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:44.510 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:44.511 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:44.538 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:44.552+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:44.612 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:44.613 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:44.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:44.665 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:44.679+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:44.723 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:44.724 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:44.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:44.740 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:44.754+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:44.841 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:44.855+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:44.876 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:44.877 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:44.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:44.969 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:44.983+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:45.041 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.041 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.042 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:45.056+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:45.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:45.143 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:45.158+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:45.193 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.193 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:45.246 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:45.260+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:45.305 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.305 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.314 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:45.359 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:45.373+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:45.419 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.420 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:45.470 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:45.483+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:45.547 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:45.561+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:45.571 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.571 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:45.648 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:45.662+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:45.682 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.682 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:45.775 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:45.789+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:45.833 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.834 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:45.849 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:45.863+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:45.951 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:45.965+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:45.968 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 7] out_osds: [1, 0] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-04-23T23:51:45.968 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-23T23:51:45.968 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-04-23T23:51:45.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-04-23T23:51:45.989 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:45.989 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:45.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:46.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:51:46.079 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:46.093+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:46.111 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:46.111 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:46.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:46.164 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:46.178+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:46.222 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:46.223 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:46.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:46.253 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:46.267+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:46.276 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-04-23T23:51:46.290+0000 7f856ad67640 -1 mon.a@0(leader).osd e126 definitely_dead 0 2024-04-23T23:51:46.354 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:46.368+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:46.372 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:46.373 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:46.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:46.466 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:46.480+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:46.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:46.524 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:46.524 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:46.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:46.574 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:46.588+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:46.640 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:46.641 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:46.657 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:46.671+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:46.658 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:46.658 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:46.674 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:46.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:46.675 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:46.675 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:46.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:46.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:46.708 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:46.708 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:46.725 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:46.725 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:46.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:46.758 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:46.759 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:46.772+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:46.786 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:46.786 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:46.787 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:46.787 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:46.787 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:46.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:46.860 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:46.874+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 129.71213592000001, 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:46.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:46.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:46.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:46.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.363399251999994, 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.908 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:46.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.912 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.363382798999993, 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:46.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.363356057999994, 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.917 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:46.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.363337263000005, 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.363319679, 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.351624174999998, 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.930 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.351448488000003, 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:46.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.351296969000003, 2024-04-23T23:51:46.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:46.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.943 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.351014343000003, 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:46.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:46.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.350868477999995, 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:46.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.349918794000004, 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:46.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.349895900000007, 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:46.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.333880524999998, 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.333870891000004, 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:46.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.318838529999994, 2024-04-23T23:51:46.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:46.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:46.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.318817206000006, 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.978 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.318794381000004, 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:46.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.318771702000006, 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:46.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:46.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:46.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.316469979000004, 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:46.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:46.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:46.993 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.0. 2024-04-23T23:51:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:51:46.995 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:47.008+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:46.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:47.001 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-04-23T23:51:47.095 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:47.110+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:47.097 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:47.098 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:47.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:47.196 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:47.210+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:47.209 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:47.209 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:47.209 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:47.209 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:47.209 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:47.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:47.297 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:47.311+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 75.260081103999994, 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 75.257486772999997, 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145831333999993, 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145812894000002, 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145793788000006, 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145775279000006, 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145672230000002, 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145652611000003, 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145547136000005, 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145521951000006, 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145488405999998, 2024-04-23T23:51:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145464642999997, 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145443602, 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145375367, 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145350898999993, 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145325201000006, 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145303435000002, 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145282968999993, 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145205110000006, 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.145175459000001, 2024-04-23T23:51:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:47.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:47.460 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:47.473+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:47.560 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:47.574+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:47.562 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:121203 3.1b 3.184d127b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:46.995333+0000", 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.58190770300000005, 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.58192791399999999, 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for peered", 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 121203 2024-04-23T23:51:47.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995333+0000", 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995333+0000", 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995335+0000", 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995342+0000", 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2810000000000003e-06 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995358+0000", 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6311999999999999e-05 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995368+0000", 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0629999999999995e-06 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997691+0000", 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023239490000000001 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:47.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997701+0000", 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1910000000000006e-06 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:121204 3.1b 3.184d127b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.58184426, 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.58192006799999996, 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for peered", 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 121204 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6400000000000003e-07 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995401+0000", 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995406+0000", 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8330000000000001e-06 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995410+0000", 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5439999999999999e-06 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997726+0000", 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023165389999999998 2024-04-23T23:51:47.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997732+0000", 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8690000000000001e-06 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 2 2024-04-23T23:51:47.571 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:47.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:47.661 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:47.675+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:47.674 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:47.674 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:47.674 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:47.674 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:47.674 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:47.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:47.769 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:47.783+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:47.829 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:47.829 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.026584076000006, 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:47.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.024540092999999, 2024-04-23T23:51:47.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:47.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:47.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.022562844999996, 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.022532212000002, 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.020589361999996, 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.018504152999995, 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:47.852 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.016494722000004, 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42084 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.014594754000001, 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050709780000000003, 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42084 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.828150+0000", 2024-04-23T23:51:47.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829969+0000", 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018184169999999999 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829974+0000", 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6659999999999999e-06 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829983+0000", 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.829989+0000", 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878699+0000", 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048709809 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878781+0000", 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2245999999999998e-05 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.878860+0000", 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9126000000000001e-05 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:47.865 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.010546226000002, 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.008501550000005, 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:47.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.006427977000001, 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.004539820999995, 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.004519044000006, 2024-04-23T23:51:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.996527549999996, 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.994592568000002, 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.994566473000006, 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62536 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.612849009000001, 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050696712999999997, 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62536 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229896+0000", 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229897+0000", 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229901+0000", 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7560000000000001e-06 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229907+0000", 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6950000000000003e-06 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229913+0000", 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2449999999999998e-06 2024-04-23T23:51:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280290+0000", 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050377078999999998 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280379+0000", 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8887000000000005e-05 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280593+0000", 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021439899999999999 2024-04-23T23:51:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.612806590000005, 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.612770632999997, 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.612746275000006, 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:47.918 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:47.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:47.919 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:47.933+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:48.020 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:48.034+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:48.063 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:48.063 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:48.063 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:48.063 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:48.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:48.121 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:48.135+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:48.214 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:48.215 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:48.215 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:48.215 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:48.215 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:48.222 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:48.237+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:48.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:48.323 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:48.336+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.827201489000004, 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.826735502000005, 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.826228952999998, 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.825933044999999, 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.825746031999998, 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.825544362000002, 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.8254041, 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.825158364999993, 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.824880840000006, 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.824638268000001, 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.824459903999994, 2024-04-23T23:51:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.824177313000007, 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.823984828999997, 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.823556887999999, 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.823404554999996, 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.823301471999997, 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.822984525999999, 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.822743098999993, 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.822518552000005, 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.822263655, 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.419 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:48.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:48.461 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:48.475+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:48.524 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:48.538+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:48.563 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:48.563 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:48.563 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:48.563 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:48.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:48.625 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:48.639+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:48.715 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:48.715 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:48.715 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:48.715 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:48.715 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:48.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:48.725 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:48.740+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:48.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T23:51:48.826 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:48.840+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:48.830 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:48.830 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:48.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:48.830 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:48.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 107.383996529, 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:48.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.574490033999993, 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.524695147000003, 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:48.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:48.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:48.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.478070079000005, 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:48.846 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:48.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.475299704999998, 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:48.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:48.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.379913876000003, 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.377495793999998, 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:48.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.658748524000003, 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.483165073000002, 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1451 3.9 e95/22 3:9566a16b:::rbd_data.117911508d31.0000000000000023:1c v 95'434, mlcod=91'429)", 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.884914008000003, 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043290560000000004, 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.68e-07 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958651+0000", 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958664+0000", 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3604000000000001e-05 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958671+0000", 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958719+0000", 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7568000000000003e-05 2024-04-23T23:51:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958742+0000", 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3532000000000001e-05 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962919+0000", 2024-04-23T23:51:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041768550000000002 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962974+0000", 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4691999999999997e-05 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1375 3.0 e96/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:1b v 96'269, mlcod=95'264)", 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.883294634999999, 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054529929999999997, 2024-04-23T23:51:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960265+0000", 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.61e-07 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960271+0000", 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2750000000000001e-06 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960282+0000", 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056000000000001e-05 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960289+0000", 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7660000000000001e-06 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960552+0000", 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026309199999999998 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960571+0000", 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8980000000000001e-05 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965676+0000", 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051052739999999999 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965717+0000", 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0788999999999999e-05 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1482 3.9 e96/22 3:90d97e73:::rbd_data.11798ae5b3f1.0000000000000008:1b v 96'436, mlcod=95'432)", 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.879196086, 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045767280000000004, 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964368+0000", 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964386+0000", 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7876e-05 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964392+0000", 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964440+0000", 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7321999999999999e-05 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964459+0000", 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9313e-05 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968889+0000", 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004429962 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968939+0000", 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0501000000000003e-05 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1378 3.0 e96/22 3:03a7baf4:::rbd_data.11798ae5b3f1.000000000000024d:1b v 96'271, mlcod=96'267)", 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.876833040999998, 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007336375, 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966731+0000", 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.011e-06 2024-04-23T23:51:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966740+0000", 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9579999999999996e-06 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966745+0000", 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966788+0000", 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985000000000003e-05 2024-04-23T23:51:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966803+0000", 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974010+0000", 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072070320000000004 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974062+0000", 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1805999999999998e-05 2024-04-23T23:51:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1483 3.9 e96/22 3:94dae484:::rbd_data.11798ae5b3f1.000000000000006c:1b v 96'438, mlcod=95'432)", 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.875903012999999, 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089730109999999995, 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967662+0000", 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967675+0000", 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2763e-05 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967683+0000", 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5530000000000004e-06 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967734+0000", 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967756+0000", 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1906999999999999e-05 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976580+0000", 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088237190000000007 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976629+0000", 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8948e-05 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1486 3.9 e96/22 3:96d7965c:::rbd_data.11798ae5b3f1.00000000000003a1:1b v 96'442, mlcod=96'436)", 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.863458999000002, 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092063349999999995, 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.893 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:51:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980107+0000", 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4180000000000002e-06 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980117+0000", 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0565e-05 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980124+0000", 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2024-04-23T23:51:48.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980185+0000", 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980206+0000", 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1472e-05 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989259+0000", 2024-04-23T23:51:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090529410000000001 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989306+0000", 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950000000000003e-05 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1383 3.4 e96/22 3:212acb2d:::rbd_data.11798ae5b3f1.0000000000000184:1b v 96'397, mlcod=96'393)", 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.857767658, 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043511569999999996, 2024-04-23T23:51:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985792+0000", 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.062e-06 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985801+0000", 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985819+0000", 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-05 2024-04-23T23:51:48.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985827+0000", 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8599999999999999e-06 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985877+0000", 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790999999999998e-05 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985900+0000", 2024-04-23T23:51:48.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2418000000000001e-05 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990100+0000", 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042000819999999999 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990142+0000", 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2469999999999998e-05 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1384 3.4 e96/22 3:25b57ffa:::rbd_data.11798ae5b3f1.000000000000006e:1b v 96'399, mlcod=96'393)", 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.857109244, 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045474490000000003, 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986450+0000", 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986457+0000", 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9809999999999997e-06 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986473+0000", 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6016e-05 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986481+0000", 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986530+0000", 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8773000000000002e-05 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986550+0000", 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0347999999999999e-05 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990945+0000", 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043951370000000003 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990997+0000", 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1674e-05 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119499 3.0 3:01c0afba:::rbd_data.1179c595b6df.0000000000000004:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.796574631999999, 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094379389999999994, 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119499 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046985+0000", 2024-04-23T23:51:48.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4079999999999999e-06 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046991+0000", 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046999+0000", 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1990000000000006e-06 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047007+0000", 2024-04-23T23:51:48.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3820000000000004e-06 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047532+0000", 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000524472 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047547+0000", 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4793e-05 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054754+0000", 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072074249999999999 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054959+0000", 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000205136 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055054+0000", 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4697999999999994e-05 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:48.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055940+0000", 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088648299999999998 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056318+0000", 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037777699999999998 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056354+0000", 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6106999999999998e-05 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056422+0000", 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7639999999999996e-05 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119508 3.0 3:05062586:::rbd_data.1179c595b6df.000000000000000d:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.794381135999998, 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007607613, 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119508 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049179+0000", 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.125e-06 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049184+0000", 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6579999999999999e-06 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049195+0000", 2024-04-23T23:51:48.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0736e-05 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049203+0000", 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049260+0000", 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6839999999999998e-05 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049270+0000", 2024-04-23T23:51:48.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.893e-06 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055559+0000", 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062896660000000002 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055647+0000", 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7496000000000004e-05 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055726+0000", 2024-04-23T23:51:48.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8917000000000004e-05 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056225+0000", 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049874700000000001 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056601+0000", 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037684200000000001 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056663+0000", 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1779000000000006e-05 2024-04-23T23:51:48.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056785+0000", 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122019 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119713 3.0 3:0770b89d:::rbd_header.1179129e5779:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.783108588999999, 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050942269999999998, 2024-04-23T23:51:48.918 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119713 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060452+0000", 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3099999999999999e-06 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060462+0000", 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6800000000000005e-06 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060485+0000", 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2864999999999999e-05 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060496+0000", 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1287999999999999e-05 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060567+0000", 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0324e-05 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060573+0000", 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8719999999999996e-06 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063243+0000", 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002669158 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063352+0000", 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109819 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063761+0000", 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040853800000000001 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.064740+0000", 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097912600000000008 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065441+0000", 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070055300000000003 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065500+0000", 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9769000000000003e-05 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065544+0000", 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3924999999999997e-05 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:48.924 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:48.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:48.926 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:48.926 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:48.927 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:48.927 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:48.927 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:48.927 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:48.928 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:48.942+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:48.941 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:48.941 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:48.973 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:48.996 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:48.996 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:49.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-23T23:51:49.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:49.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:49.029 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:49.043+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:49.041 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:49.054 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.054 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:49.130 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:49.144+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:49.168 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.168 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:49.260 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:49.274+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:49.329 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.329 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.332 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:49.346+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:49.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:49.433 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:49.447+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:49.489 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.489 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:49.544 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:49.558+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:49.602 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.602 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:49.635 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:49.649+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:49.735 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:49.749+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:49.755 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.755 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:49.837 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-04-23T23:51:49.851+0000 7f6fc0c4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32298) UID: 0 2024-04-23T23:51:49.867 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:49.867 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:49.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:49.938 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:49.952+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-04-23T23:51:50.025 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.025 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:50.046 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-23T23:51:50.060+0000 7fc006483640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32496) UID: 0 2024-04-23T23:51:50.135 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.136 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.140 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:50.154+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:50.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:50.240 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-04-23T23:51:50.254+0000 7f5a2acf4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32291) UID: 0 2024-04-23T23:51:50.245 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.246 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:50.342 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-23T23:51:50.356+0000 7f7d2cb57640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32495) UID: 0 2024-04-23T23:51:50.398 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.398 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:50.451 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:50.465+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:50.510 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.510 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:50.560 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:50.574+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:50.624 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.624 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:50.677 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:50.691+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:50.737 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.737 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:50.746 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-04-23T23:51:50.760+0000 7fa30c59f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32297) UID: 0 2024-04-23T23:51:50.847 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-04-23T23:51:50.861+0000 7fe711962640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32293) UID: 0 2024-04-23T23:51:50.888 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:50.888 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:50.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-23T23:51:50.948 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-23T23:51:50.962+0000 7efdc8491640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 46936) UID: 0 2024-04-23T23:51:51.010 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:51.010 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:51.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-23T23:51:51.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T23:51:51.061 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-23T23:51:51.075+0000 7f58172f4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32502) UID: 0 2024-04-23T23:51:51.077 INFO:teuthology.orchestra.run.smithi092.stdout:All operations completed A-OK! 2024-04-23T23:51:51.111 DEBUG:teuthology.parallel:result is None 2024-04-23T23:51:51.112 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-04-23T23:51:51.122 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-23T23:51:51.186 INFO:tasks.thrashosds:joining thrashosds 2024-04-23T23:51:51.188 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-23T23:51:51.188 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-23T23:51:51.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-23T23:51:51.189 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-04-23T23:51:51.189 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-23T23:51:51.203 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-04-23T23:51:51.203 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-23T23:51:51.220 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-04-23T23:51:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-23T23:51:51.237 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-04-23T23:51:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-23T23:51:51.255 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-04-23T23:51:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-23T23:51:51.272 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-04-23T23:51:51.272 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-23T23:51:51.289 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-04-23T23:51:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42018 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1994752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.491823820999997, 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050775422000000001, 2024-04-23T23:51:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42018 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.816161+0000", 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817969+0000", 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001807616 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817973+0000", 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817985+0000", 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007e-05 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.817992+0000", 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0210000000000002e-06 2024-04-23T23:51:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866775+0000", 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048782450999999998 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866849+0000", 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4148999999999998e-05 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.866937+0000", 2024-04-23T23:51:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7708999999999997e-05 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42028 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2015232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.489779838000004, 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050778385000000002, 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42028 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.818205+0000", 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819969+0000", 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00176374 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819974+0000", 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2379999999999997e-06 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819985+0000", 2024-04-23T23:51:51.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1158000000000001e-05 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.819992+0000", 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868810+0000", 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048817583999999997 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868887+0000", 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7330999999999995e-05 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.868984+0000", 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42037 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1550336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.487802590000001, 2024-04-23T23:51:51.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050755055, 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42037 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820182+0000", 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820183+0000", 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820189+0000", 2024-04-23T23:51:51.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820200+0000", 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820205+0000", 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3260000000000003e-06 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870780+0000", 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050574727999999999 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870864+0000", 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3914999999999999e-05 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870937+0000", 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3407999999999999e-05 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42038 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2035712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.487771957000007, 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951021999999999, 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42038 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.820213+0000", 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821970+0000", 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001756908 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821975+0000", 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0309999999999998e-06 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821985+0000", 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0343000000000001e-05 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.821991+0000", 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.870981+0000", 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048990432 2024-04-23T23:51:51.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871067+0000", 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5896999999999994e-05 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.871164+0000", 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6904999999999995e-05 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42048 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.485829107000001, 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050784956999999999, 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42048 2024-04-23T23:51:51.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.822156+0000", 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823969+0000", 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018127989999999999 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823974+0000", 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823989+0000", 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5588000000000001e-05 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.823997+0000", 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872812+0000", 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048815002000000003 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872884+0000", 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2488000000000001e-05 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.872941+0000", 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6576000000000002e-05 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42058 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.483743898, 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050749225000000002, 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42058 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.824241+0000", 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825972+0000", 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730592 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825976+0000", 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6e-06 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825987+0000", 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1051e-05 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.825994+0000", 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874787+0000", 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048792993999999999 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874877+0000", 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0127e-05 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.874990+0000", 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113021 2024-04-23T23:51:51.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42071 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.481734466999995, 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050721872000000001, 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42071 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.826251+0000", 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827978+0000", 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001727981 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827983+0000", 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3660000000000002e-06 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827992+0000", 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6250000000000002e-06 2024-04-23T23:51:51.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.827999+0000", 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7259999999999996e-06 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876855+0000", 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048855841999999997 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876911+0000", 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5745000000000002e-05 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.876972+0000", 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1587000000000005e-05 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42118 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1738752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.475785970999993, 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050743165, 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42118 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.832199+0000", 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833973+0000", 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001774496 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833979+0000", 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833992+0000", 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3309000000000001e-05 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.833998+0000", 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7930000000000003e-06 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882772+0000", 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048773874000000002 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882844+0000", 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2446000000000004e-05 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.882942+0000", 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130999999999999e-05 2024-04-23T23:51:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42137 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.473741294999996, 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050779042000000003, 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42137 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.834244+0000", 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835971+0000", 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017276240000000001 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835976+0000", 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835987+0000", 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1338999999999999e-05 2024-04-23T23:51:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.835995+0000", 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5979999999999999e-06 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884872+0000", 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048877084000000001 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.884929+0000", 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6422999999999998e-05 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.885023+0000", 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4160000000000006e-05 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42156 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1828864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.471667722000007, 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050878461999999999, 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42156 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.836317+0000", 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837969+0000", 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016515360000000001 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837977+0000", 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0360000000000006e-06 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837989+0000", 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2303000000000001e-05 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.837998+0000", 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2620000000000004e-06 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887013+0000", 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049015050999999997 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887091+0000", 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7550000000000001e-05 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.887196+0000", 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104724 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42172 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.469779566, 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050718721000000001, 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42172 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838205+0000", 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838206+0000", 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838208+0000", 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2199999999999999e-06 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838214+0000", 2024-04-23T23:51:51.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7570000000000001e-06 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838219+0000", 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5639999999999996e-06 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888813+0000", 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050594055999999998 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888869+0000", 2024-04-23T23:51:51.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5886e-05 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888924+0000", 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4790999999999998e-05 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42174 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 1869824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:51.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.469758788999997, 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050912150000000003, 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42174 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.838226+0000", 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839968+0000", 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001741771 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839973+0000", 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839982+0000", 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2569999999999996e-06 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.839989+0000", 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5579999999999997e-06 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.888955+0000", 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048965920000000003 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889035+0000", 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0181999999999999e-05 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.889138+0000", 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103628 2024-04-23T23:51:51.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42254 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.461767295000001, 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050867680999999998, 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42254 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.846218+0000", 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847967+0000", 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017495729999999999 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847972+0000", 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847984+0000", 2024-04-23T23:51:51.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1722999999999999e-05 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.847991+0000", 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896909+0000", 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048918221999999997 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.896990+0000", 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1314000000000001e-05 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.897085+0000", 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5265000000000004e-05 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42306 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.459832313000007, 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050805523999999998, 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42306 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848153+0000", 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848156+0000", 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.537e-06 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848163+0000", 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0509999999999999e-06 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848168+0000", 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898774+0000", 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050606847000000003 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898853+0000", 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8558000000000002e-05 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.898958+0000", 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105246 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:42308 3.1f 3:fd0a858a:::rbd_data.11796b6600d0.0000000000000000:15 [read 2074624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.459806217999997, 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050951327999999997, 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42308 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.848179+0000", 2024-04-23T23:51:51.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849968+0000", 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017888960000000001 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849972+0000", 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849984+0000", 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1314999999999999e-05 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.849991+0000", 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899008+0000", 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049017803999999998 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899050+0000", 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2104999999999998e-05 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:24.899130+0000", 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9637999999999999e-05 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.363 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62539 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.078046334999996, 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053123500999999997, 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62539 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229939+0000", 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229942+0000", 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2030000000000001e-06 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229950+0000", 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7090000000000002e-06 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229956+0000", 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280623+0000", 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050667550999999998 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.280730+0000", 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106193 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283062+0000", 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023325009999999998 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62541 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.078010378000002, 2024-04-23T23:51:51.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054099080000000001, 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62541 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:51.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229975+0000", 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229978+0000", 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7539999999999998e-06 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229985+0000", 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2230000000000001e-06 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229991+0000", 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283096+0000", 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053104599000000002 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.283173+0000", 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7004999999999999e-05 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284074+0000", 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090090099999999996 2024-04-23T23:51:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62544 3.6 3:67eab873:::rbd_data.117966a158ce.000000000000000b:19 [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.077986019999997, 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055624008000000003, 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62544 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229999+0000", 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.72e-07 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230002+0000", 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2019999999999998e-06 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230008+0000", 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.230013+0000", 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1429999999999997e-06 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284101+0000", 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054088002000000003 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.284184+0000", 2024-04-23T23:51:51.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3020000000000001e-05 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.285623+0000", 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001439011 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:121203 3.1b 3.184d127b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:46.995333+0000", 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3126516380000002, 2024-04-23T23:51:51.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96394724899999995, 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 121203 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995333+0000", 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995333+0000", 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995335+0000", 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995342+0000", 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2810000000000003e-06 2024-04-23T23:51:51.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995358+0000", 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6311999999999999e-05 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995368+0000", 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0629999999999995e-06 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997691+0000", 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023239490000000001 2024-04-23T23:51:51.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997701+0000", 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1910000000000006e-06 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:47.959270+0000", 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96156907000000003 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:47.959281+0000", 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0845999999999999e-05 2024-04-23T23:51:51.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:121204 3.1b 3.184d127b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.312588195, 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96391934499999998, 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:51.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 121204 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:51.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995397+0000", 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6400000000000003e-07 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995401+0000", 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995406+0000", 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8330000000000001e-06 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.995410+0000", 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5439999999999999e-06 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997726+0000", 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023165389999999998 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:46.997732+0000", 2024-04-23T23:51:51.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8690000000000001e-06 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:47.959308+0000", 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96157552899999998 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:47.959316+0000", 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5900000000000008e-06 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.388 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:51.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-23T23:51:51.389 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-04-23T23:51:51.390 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-23T23:51:51.533 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:51.533 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:51.533 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:51.533 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:51.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-23T23:51:51.644 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:51.645 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:51.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-23T23:51:51.758 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:51.758 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49200 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 835584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.258258499999997, 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061085471000000002, 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.759 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49200 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513344+0000", 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513537+0000", 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192802 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513541+0000", 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513550+0000", 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7960000000000001e-06 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513558+0000", 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5939999999999994e-06 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574200+0000", 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060642026000000002 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574279+0000", 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8874000000000004e-05 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574430+0000", 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015034599999999999 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49202 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 839680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.257792512999998, 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066755, 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49202 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.513810+0000", 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514093+0000", 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028329199999999998 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514097+0000", 2024-04-23T23:51:51.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-06 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514105+0000", 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514112+0000", 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574662+0000", 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060550233000000002 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574756+0000", 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3894000000000006e-05 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.574877+0000", 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121029 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49204 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.257285964000005, 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061131360000000003, 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49204 2024-04-23T23:51:51.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514317+0000", 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514586+0000", 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026964900000000001 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514590+0000", 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.427e-06 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514598+0000", 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514603+0000", 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8270000000000003e-06 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575254+0000", 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060651008999999999 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575340+0000", 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5642000000000004e-05 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575448+0000", 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108012 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49205 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 847872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.256990056000006, 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061066533999999999, 2024-04-23T23:51:51.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49205 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514613+0000", 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514773+0000", 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159999 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514776+0000", 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514785+0000", 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1389999999999995e-06 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514791+0000", 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.032e-06 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575490+0000", 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060699477000000002 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575570+0000", 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0227000000000003e-05 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575679+0000", 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108885 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49206 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.256803043000005, 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061067655999999998, 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49206 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514800+0000", 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514969+0000", 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168927 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514977+0000", 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5099999999999998e-06 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514985+0000", 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0490000000000004e-06 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.514992+0000", 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5390000000000004e-06 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575718+0000", 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060726128999999997 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575774+0000", 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831000000000001e-05 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575867+0000", 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3671000000000005e-05 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49207 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:51.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.256601372999995, 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061146084000000003, 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49207 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:51.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515001+0000", 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515113+0000", 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112005 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515117+0000", 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5860000000000001e-06 2024-04-23T23:51:51.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515125+0000", 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515132+0000", 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.575966+0000", 2024-04-23T23:51:51.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060834284000000002 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576021+0000", 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049000000000001e-05 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576147+0000", 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012630900000000001 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49208 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.256461110999993, 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061239637, 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49208 2024-04-23T23:51:51.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515142+0000", 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515352+0000", 2024-04-23T23:51:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021059599999999999 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515357+0000", 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.352e-06 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515368+0000", 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1018e-05 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515375+0000", 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3860000000000001e-06 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576190+0000", 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060814646999999999 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576271+0000", 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1457000000000003e-05 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576381+0000", 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011018099999999999 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49209 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 856064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.256215376, 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061289030000000001, 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49209 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515387+0000", 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515632+0000", 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024510299999999999 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515637+0000", 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2470000000000002e-06 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515646+0000", 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515653+0000", 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.618e-06 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576449+0000", 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060796663000000001 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576529+0000", 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9956999999999994e-05 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576676+0000", 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014711499999999999 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49210 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 489472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.255937850999999, 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061262998999999999, 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49210 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515665+0000", 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515872+0000", 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020745200000000001 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515877+0000", 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515887+0000", 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8209999999999997e-06 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515895+0000", 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576744+0000", 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060848878000000002 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576808+0000", 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4406000000000006e-05 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576928+0000", 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011937 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49211 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.255695278999994, 2024-04-23T23:51:51.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061291469000000001, 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49211 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.515907+0000", 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516058+0000", 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015029400000000001 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516062+0000", 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516069+0000", 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0140000000000001e-06 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516075+0000", 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.576981+0000", 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060905966999999998 2024-04-23T23:51:51.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577065+0000", 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4067999999999995e-05 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577199+0000", 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133996 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49212 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.255516915000001, 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061344450000000002, 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49212 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516086+0000", 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516345+0000", 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025967299999999998 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516349+0000", 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7519999999999998e-06 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516356+0000", 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516360+0000", 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999996e-06 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577238+0000", 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060877531999999998 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577314+0000", 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6098000000000002e-05 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577430+0000", 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116434 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49213 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.255234324, 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061296244999999999, 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49213 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516368+0000", 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516532+0000", 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016375800000000001 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516535+0000", 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516546+0000", 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0963e-05 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516552+0000", 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577481+0000", 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060929189000000002 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577545+0000", 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3979999999999994e-05 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577665+0000", 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011919 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49214 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.255041840000004, 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061345838, 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49214 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516561+0000", 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516967+0000", 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040633100000000001 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516971+0000", 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7890000000000001e-06 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516977+0000", 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-04-23T23:51:51.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516981+0000", 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1649999999999999e-06 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577701+0000", 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060720518000000001 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577778+0000", 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6631999999999994e-05 2024-04-23T23:51:51.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577907+0000", 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012855899999999999 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49215 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 868352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.254613899000006, 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061189355000000001, 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49215 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.516989+0000", 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517105+0000", 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115989 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517109+0000", 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2570000000000001e-06 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517121+0000", 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517129+0000", 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1380000000000007e-06 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.577993+0000", 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060863610999999998 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578081+0000", 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7805000000000004e-05 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578178+0000", 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7501e-05 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49216 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 501760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:51.819 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.254461566000003, 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061196307999999998, 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49216 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517141+0000", 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517220+0000", 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8399000000000006e-05 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517223+0000", 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3000000000000002e-06 2024-04-23T23:51:51.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517228+0000", 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1189999999999996e-06 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517233+0000", 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3700000000000003e-06 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578208+0000", 2024-04-23T23:51:51.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060975192999999997 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578265+0000", 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6038000000000001e-05 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578337+0000", 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2889e-05 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49217 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 872448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.254358483000004, 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061279911999999999, 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49217 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517244+0000", 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517535+0000", 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000291126 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517539+0000", 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0829999999999997e-06 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517547+0000", 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2740000000000002e-06 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517552+0000", 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-04-23T23:51:51.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578377+0000", 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060825552999999997 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578446+0000", 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8847e-05 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578524+0000", 2024-04-23T23:51:51.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7991000000000006e-05 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49218 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.254041537000006, 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061149896000000002, 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49218 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517561+0000", 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517769+0000", 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000207925 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517776+0000", 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1049999999999997e-06 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517784+0000", 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:51.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517790+0000", 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000003e-06 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578558+0000", 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060767947000000003 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578624+0000", 2024-04-23T23:51:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5654999999999994e-05 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578711+0000", 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7379000000000005e-05 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49219 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.25380011, 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061145465000000003, 2024-04-23T23:51:51.832 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49219 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517803+0000", 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517990+0000", 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018743500000000001 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.517994+0000", 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518009+0000", 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4620000000000001e-05 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518016+0000", 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0600000000000002e-06 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578739+0000", 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060722951999999997 2024-04-23T23:51:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578805+0000", 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6328000000000006e-05 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.578948+0000", 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014270999999999999 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.836 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49220 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 509952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.253575562999998, 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061095700000000003, 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49220 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518027+0000", 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518250+0000", 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022272200000000001 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518254+0000", 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518265+0000", 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518271+0000", 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3679999999999998e-06 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579001+0000", 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060730315999999999 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579046+0000", 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4413000000000003e-05 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579123+0000", 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7083999999999996e-05 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:49221 3.19 3:9b7876f4:::rbd_data.11796b6600d0.0000000000000002:15 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.253320665999993, 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061097857999999998, 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49221 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:51.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518282+0000", 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518657+0000", 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037474300000000003 2024-04-23T23:51:51.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518661+0000", 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518672+0000", 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.518678+0000", 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4559999999999997e-06 2024-04-23T23:51:51.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579164+0000", 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060485412000000002 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579242+0000", 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8818000000000003e-05 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:25.579380+0000", 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013755700000000001 2024-04-23T23:51:51.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:51.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:51.845 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:51.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-23T23:51:51.948 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:51.948 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:51.948 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:51.949 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:51.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-23T23:51:52.002 INFO:tasks.thrashosds.thrasher:inning osd 2024-04-23T23:51:52.002 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-04-23T23:51:52.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-04-23T23:51:52.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-23T23:51:52.062 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:52.063 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:52.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:3037 3.4 e32/22 3:24dff012:::rbd_data.11798eaf3f6b.0000000000000002:head v 32'57, mlcod=31'54)", 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 110.77821224900001, 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006141606, 2024-04-23T23:51:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459562+0000", 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459563+0000", 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459570+0000", 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3950000000000003e-06 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459578+0000", 2024-04-23T23:51:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4449999999999996e-06 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459586+0000", 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459643+0000", 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7398999999999997e-05 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.459663+0000", 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439000000000001e-05 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465652+0000", 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059888900000000002 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:01.465704+0000", 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2225999999999998e-05 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:62791 3.4 e58/22 3:21c77aed:::rbd_data.11798ae5b3f1.0000000000000020:head v 58'119, mlcod=55'115)", 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.968705753999998, 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041073919999999996, 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269069+0000", 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269074+0000", 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9350000000000002e-06 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269081+0000", 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3490000000000003e-06 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269087+0000", 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.395e-06 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269591+0000", 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050436899999999995 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.269612+0000", 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0434000000000001e-05 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273135+0000", 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035236439999999998 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.273176+0000", 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893000000000003e-05 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63230 3.0 e58/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:head v 58'144, mlcod=55'142)", 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.918910866999994, 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042983789999999997, 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318864+0000", 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.318865+0000", 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0240000000000001e-06 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319033+0000", 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016865599999999999 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319052+0000", 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9137e-05 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319061+0000", 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319126+0000", 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4331000000000005e-05 2024-04-23T23:51:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.319171+0000", 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-05 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323118+0000", 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039469420000000002 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.323162+0000", 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4409999999999997e-05 2024-04-23T23:51:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63574 3.0 e58/22 3:0611b68f:::rbd_data.11798ae5b3f1.0000000000000053:head v 58'149, mlcod=58'146)", 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.872285798999997, 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004587977, 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365489+0000", 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365495+0000", 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365508+0000", 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3173000000000001e-05 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365515+0000", 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2349999999999996e-06 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365571+0000", 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6521000000000003e-05 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.365591+0000", 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.997e-05 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370039+0000", 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044474659999999997 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.370077+0000", 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8073e-05 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:63581 3.0 e58/22 3:0506907a:::rbd_data.11798ae5b3f1.0000000000000054:head v 58'150, mlcod=58'147)", 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.869515425000003, 2024-04-23T23:51:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047275729999999997, 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368259+0000", 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368260+0000", 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4099999999999998e-07 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368267+0000", 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3320000000000003e-06 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368280+0000", 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368287+0000", 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8940000000000004e-06 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368331+0000", 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4373999999999999e-05 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.368348+0000", 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7144999999999999e-05 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372938+0000", 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045900089999999999 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.372987+0000", 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64076 3.4 e58/22 3:23f89a27:::rbd_data.11798ae5b3f1.00000000000000b4:head v 58'143, mlcod=58'139)", 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.774129595999995, 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048665829999999998, 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463645+0000", 2024-04-23T23:51:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463651+0000", 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4419999999999997e-06 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463660+0000", 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5310000000000007e-06 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463666+0000", 2024-04-23T23:51:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463712+0000", 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5188999999999997e-05 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.463729+0000", 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6943e-05 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468472+0000", 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047435919999999996 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.468511+0000", 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9229e-05 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:64094 3.4 e58/22 3:24373c06:::rbd_data.11798ae5b3f1.00000000000000c1:head v 58'145, mlcod=58'139)", 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.771711514000003, 2024-04-23T23:51:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051218210000000004, 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466063+0000", 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466064+0000", 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466069+0000", 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7169999999999996e-06 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466082+0000", 2024-04-23T23:51:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2288e-05 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466088+0000", 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.613e-06 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466136+0000", 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7518999999999997e-05 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.466155+0000", 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9249000000000002e-05 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471138+0000", 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049827839999999997 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.471185+0000", 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950999999999998e-05 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107458 3.9 e77/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 77'405, mlcod=77'402)", 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.052964244000002, 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012018977, 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184810+0000", 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.184812+0000", 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6249999999999999e-06 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191517+0000", 2024-04-23T23:51:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067051649999999999 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191527+0000", 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0497e-05 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191534+0000", 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191586+0000", 2024-04-23T23:51:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2030000000000002e-05 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.191609+0000", 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2169999999999999e-05 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196742+0000", 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051339209999999996 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:57.196829+0000", 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6679000000000001e-05 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4473.0:107670 3.9 e79/22 3:96020db9:::rbd_data.1179ac1b9b5c.0000000000000001:head v 79'408, mlcod=77'405)", 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.877380793, 2024-04-23T23:51:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052820879999999999, 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360394+0000", 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360401+0000", 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9709999999999998e-06 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360413+0000", 2024-04-23T23:51:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1664e-05 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360420+0000", 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2640000000000002e-06 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360476+0000", 2024-04-23T23:51:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6465000000000002e-05 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.360499+0000", 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2269e-05 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365629+0000", 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051302600000000002 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:58.365676+0000", 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6717e-05 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1451 3.9 e95/22 3:9566a16b:::rbd_data.117911508d31.0000000000000023:1c v 95'434, mlcod=91'429)", 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.279129728000001, 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043290560000000004, 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958645+0000", 2024-04-23T23:51:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.68e-07 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958651+0000", 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958664+0000", 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3604000000000001e-05 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958671+0000", 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958719+0000", 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7568000000000003e-05 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.958742+0000", 2024-04-23T23:51:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3532000000000001e-05 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962919+0000", 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041768550000000002 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:14.962974+0000", 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4691999999999997e-05 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1375 3.0 e96/22 3:03464641:::rbd_data.11798ae5b3f1.0000000000000034:1b v 96'269, mlcod=95'264)", 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.277510354999997, 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054529929999999997, 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960264+0000", 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960265+0000", 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.61e-07 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960271+0000", 2024-04-23T23:51:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2750000000000001e-06 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960282+0000", 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056000000000001e-05 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960289+0000", 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7660000000000001e-06 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960552+0000", 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026309199999999998 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.960571+0000", 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8980000000000001e-05 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965676+0000", 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051052739999999999 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.965717+0000", 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0788999999999999e-05 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1482 3.9 e96/22 3:90d97e73:::rbd_data.11798ae5b3f1.0000000000000008:1b v 96'436, mlcod=95'432)", 2024-04-23T23:51:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.273411805999999, 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045767280000000004, 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964363+0000", 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964368+0000", 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964386+0000", 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7876e-05 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964392+0000", 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964440+0000", 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7321999999999999e-05 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.964459+0000", 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9313e-05 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968889+0000", 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004429962 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.968939+0000", 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0501000000000003e-05 2024-04-23T23:51:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1378 3.0 e96/22 3:03a7baf4:::rbd_data.11798ae5b3f1.000000000000024d:1b v 96'271, mlcod=96'267)", 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.271048761000003, 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007336375, 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966726+0000", 2024-04-23T23:51:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966731+0000", 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.011e-06 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966740+0000", 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9579999999999996e-06 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966745+0000", 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966788+0000", 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985000000000003e-05 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.966803+0000", 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974010+0000", 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072070320000000004 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.974062+0000", 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1805999999999998e-05 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1483 3.9 e96/22 3:94dae484:::rbd_data.11798ae5b3f1.000000000000006c:1b v 96'438, mlcod=95'432)", 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.270118732999997, 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089730109999999995, 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967656+0000", 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967662+0000", 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0929999999999999e-06 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967675+0000", 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2763e-05 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967683+0000", 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5530000000000004e-06 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967734+0000", 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.967756+0000", 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1906999999999999e-05 2024-04-23T23:51:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976580+0000", 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088237190000000007 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.976629+0000", 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8948e-05 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.4.0:1486 3.9 e96/22 3:96d7965c:::rbd_data.11798ae5b3f1.00000000000003a1:1b v 96'442, mlcod=96'436)", 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.257674719000001, 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092063349999999995, 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980100+0000", 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4099999999999998e-07 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980107+0000", 2024-04-23T23:51:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4180000000000002e-06 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980117+0000", 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0565e-05 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980124+0000", 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980185+0000", 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.980206+0000", 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1472e-05 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989259+0000", 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090529410000000001 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.989306+0000", 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6950000000000003e-05 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1383 3.4 e96/22 3:212acb2d:::rbd_data.11798ae5b3f1.0000000000000184:1b v 96'397, mlcod=96'393)", 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.251983377999998, 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043511569999999996, 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985791+0000", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985792+0000", 2024-04-23T23:51:52.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.062e-06 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985801+0000", 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985819+0000", 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-05 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985827+0000", 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8599999999999999e-06 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985877+0000", 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790999999999998e-05 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.985900+0000", 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2418000000000001e-05 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990100+0000", 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042000819999999999 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990142+0000", 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2469999999999998e-05 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.299 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.1.0:1384 3.4 e96/22 3:25b57ffa:::rbd_data.11798ae5b3f1.000000000000006e:1b v 96'399, mlcod=96'393)", 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.251324963999998, 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045474490000000003, 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986449+0000", 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986450+0000", 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1800000000000005e-07 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986457+0000", 2024-04-23T23:51:52.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9809999999999997e-06 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986473+0000", 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6016e-05 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986481+0000", 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986530+0000", 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8773000000000002e-05 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.986550+0000", 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0347999999999999e-05 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990945+0000", 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043951370000000003 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:15.990997+0000", 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1674e-05 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119499 3.0 3:01c0afba:::rbd_data.1179c595b6df.0000000000000004:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.190790352, 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094379389999999994, 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119499 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046984+0000", 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046985+0000", 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4079999999999999e-06 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046991+0000", 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.046999+0000", 2024-04-23T23:51:52.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1990000000000006e-06 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047007+0000", 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3820000000000004e-06 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047532+0000", 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000524472 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.047547+0000", 2024-04-23T23:51:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4793e-05 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054754+0000", 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072074249999999999 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.054959+0000", 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000205136 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:52.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055054+0000", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4697999999999994e-05 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055940+0000", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088648299999999998 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056318+0000", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037777699999999998 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056354+0000", 2024-04-23T23:51:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6106999999999998e-05 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056422+0000", 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7639999999999996e-05 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119508 3.0 3:05062586:::rbd_data.1179c595b6df.000000000000000d:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-04-23T23:51:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.188596856, 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007607613, 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119508 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049178+0000", 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049179+0000", 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.125e-06 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049184+0000", 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6579999999999999e-06 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049195+0000", 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0736e-05 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049203+0000", 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049260+0000", 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6839999999999998e-05 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:52.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.049270+0000", 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.893e-06 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055559+0000", 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062896660000000002 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055647+0000", 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7496000000000004e-05 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:52.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.055726+0000", 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8917000000000004e-05 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056225+0000", 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049874700000000001 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056601+0000", 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037684200000000001 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056663+0000", 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1779000000000006e-05 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:30.056785+0000", 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122019 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.317 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:119713 3.0 3:0770b89d:::rbd_header.1179129e5779:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.177324308999999, 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050942269999999998, 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 119713 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060450+0000", 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060452+0000", 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3099999999999999e-06 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060462+0000", 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6800000000000005e-06 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060485+0000", 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2864999999999999e-05 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060496+0000", 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1287999999999999e-05 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060567+0000", 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0324e-05 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.060573+0000", 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8719999999999996e-06 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063243+0000", 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002669158 2024-04-23T23:51:52.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063352+0000", 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109819 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.063761+0000", 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040853800000000001 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.064740+0000", 2024-04-23T23:51:52.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097912600000000008 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065441+0000", 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070055300000000003 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065500+0000", 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9769000000000003e-05 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:52.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:51:33.065544+0000", 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3924999999999997e-05 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:52.324 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:52.325 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-23T23:51:52.326 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-04-23T23:51:52.322+0000 7f856ad67640 -1 mon.a@0(leader).osd e131 definitely_dead 0 2024-04-23T23:51:52.440 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:52.440 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:52.440 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:52.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-23T23:51:52.553 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:52.553 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:52.553 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:52.553 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:52.553 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:52.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-23T23:51:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37781 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1443840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.246930790999997, 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044781100999999997, 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37781 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430984+0000", 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430985+0000", 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1799999999999999e-07 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430986+0000", 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9399999999999993e-07 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430989+0000", 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9189999999999999e-06 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430992+0000", 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9670000000000001e-06 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475677+0000", 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044685017 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475717+0000", 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0467999999999999e-05 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475766+0000", 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8517999999999999e-05 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37782 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.669 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.246920911000004, 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001269000000003, 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37782 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.430994+0000", 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431164+0000", 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016950799999999999 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431176+0000", 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1779e-05 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431183+0000", 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7400000000000004e-06 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431189+0000", 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0859999999999998e-06 2024-04-23T23:51:52.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475798+0000", 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044608907000000003 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475890+0000", 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1419e-05 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.475996+0000", 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010582999999999999 2024-04-23T23:51:52.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37783 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1452032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.246710563999997, 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044909080999999997, 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37783 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.431205+0000", 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432830+0000", 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001625329 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432835+0000", 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432844+0000", 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3530000000000001e-06 2024-04-23T23:51:52.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.432851+0000", 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3779999999999998e-06 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476025+0000", 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.04317435 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476065+0000", 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0268000000000001e-05 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.476114+0000", 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8643999999999997e-05 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37792 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1488896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:52.678 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.244879225000005, 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044784216000000002, 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37792 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:52.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.433036+0000", 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434828+0000", 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017921390000000001 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434833+0000", 2024-04-23T23:51:52.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7729999999999999e-06 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434841+0000", 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9240000000000007e-06 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.434846+0000", 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.079e-06 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477654+0000", 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042808205000000002 2024-04-23T23:51:52.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477734+0000", 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0177000000000002e-05 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.477820+0000", 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5919000000000002e-05 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:37804 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1538048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.242809593999993, 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044824066000000003, 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 37804 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.435106+0000", 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436830+0000", 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017246259999999999 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436835+0000", 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9400000000000001e-06 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436845+0000", 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6630000000000005e-06 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.436852+0000", 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-04-23T23:51:52.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479765+0000", 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.042913157 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479841+0000", 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6336999999999998e-05 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.479930+0000", 2024-04-23T23:51:52.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8597000000000006e-05 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44625 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730719358000002, 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044857619000000001, 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44625 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947196+0000", 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5899999999999996e-07 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947199+0000", 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.649e-06 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947209+0000", 2024-04-23T23:51:52.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947214+0000", 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4060000000000004e-06 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991959+0000", 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044745394000000001 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.991996+0000", 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6328000000000002e-05 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992054+0000", 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7739999999999999e-05 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44626 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1648640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730693701999996, 2024-04-23T23:51:52.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044953326000000002, 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44626 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:52.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947222+0000", 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4499999999999997e-07 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947224+0000", 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1770000000000001e-06 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947231+0000", 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5320000000000003e-06 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947236+0000", 2024-04-23T23:51:52.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1139999999999997e-06 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992082+0000", 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044846282000000001 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992123+0000", 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0737e-05 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992175+0000", 2024-04-23T23:51:52.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2039000000000001e-05 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44627 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730626212000004, 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045012803999999997, 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.696 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44627 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947289+0000", 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947290+0000", 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.61e-07 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947293+0000", 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2499999999999998e-06 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947302+0000", 2024-04-23T23:51:52.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9590000000000001e-06 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947308+0000", 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2809999999999997e-06 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992195+0000", 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044886316000000002 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992242+0000", 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7573e-05 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992302+0000", 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9763999999999999e-05 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44628 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1652736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730581150999996, 2024-04-23T23:51:52.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045128145000000001, 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44628 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947334+0000", 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947335+0000", 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000004e-07 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947338+0000", 2024-04-23T23:51:52.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1760000000000002e-06 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947346+0000", 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7330000000000003e-06 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947350+0000", 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7589999999999997e-06 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992335+0000", 2024-04-23T23:51:52.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044984545000000001 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992410+0000", 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4787000000000001e-05 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992462+0000", 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2475000000000002e-05 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44629 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2060288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730544651000002, 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045205956999999998, 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44629 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947371+0000", 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5900000000000003e-07 2024-04-23T23:51:52.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947373+0000", 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7910000000000001e-06 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947377+0000", 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2679999999999996e-06 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947380+0000", 2024-04-23T23:51:52.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2210000000000002e-06 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992485+0000", 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045104753999999997 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992524+0000", 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9453000000000003e-05 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992577+0000", 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2111000000000003e-05 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44630 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1656832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730527691999995, 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045401576999999999, 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44630 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947388+0000", 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0400000000000002e-07 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947389+0000", 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.26e-06 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947393+0000", 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4039999999999999e-06 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947396+0000", 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992612+0000", 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045215801999999999 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992692+0000", 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9239999999999993e-05 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992789+0000", 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7635999999999998e-05 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44631 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2064384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.713 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730449007999994, 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045442817000000003, 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44631 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:52.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947466+0000", 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947467+0000", 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8500000000000001e-07 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947469+0000", 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5739999999999998e-06 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947478+0000", 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947484+0000", 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2949999999999999e-06 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992816+0000", 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045332073000000001 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992859+0000", 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.282e-05 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992909+0000", 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9839000000000002e-05 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.717 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44632 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1660928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730381975, 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045535977999999998, 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.718 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44632 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947533+0000", 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947638+0000", 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010504400000000001 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947641+0000", 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947652+0000", 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0603e-05 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947656+0000", 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.172e-06 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992936+0000", 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045279729999999997 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.992977+0000", 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1078000000000002e-05 2024-04-23T23:51:52.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993069+0000", 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2331999999999996e-05 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44633 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2068480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.730248373999999, 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045531209000000003, 2024-04-23T23:51:52.722 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44633 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.947667+0000", 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948052+0000", 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038548199999999998 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948056+0000", 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0330000000000002e-06 2024-04-23T23:51:52.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948066+0000", 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1379999999999996e-06 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948073+0000", 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3019999999999997e-06 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993099+0000", 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045025793000000001 2024-04-23T23:51:52.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993137+0000", 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8401e-05 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993198+0000", 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1060000000000002e-05 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44634 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.729829694000003, 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045230218000000003, 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44634 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948086+0000", 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948207+0000", 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012157799999999999 2024-04-23T23:51:52.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948211+0000", 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3510000000000002e-06 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948217+0000", 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8090000000000004e-06 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948223+0000", 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5629999999999999e-06 2024-04-23T23:51:52.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993224+0000", 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045001425999999997 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993265+0000", 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0439999999999999e-05 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993316+0000", 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1051000000000003e-05 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44635 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.729679051999994, 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045186697999999997, 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44635 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948236+0000", 2024-04-23T23:51:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948339+0000", 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102808 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948342+0000", 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.886e-06 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948348+0000", 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6609999999999997e-06 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948353+0000", 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6030000000000004e-06 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993340+0000", 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044986774 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993376+0000", 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5973000000000003e-05 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993423+0000", 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6993000000000002e-05 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44636 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2076672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.735 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.729553670000001, 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045185374, 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44636 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948362+0000", 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948551+0000", 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018944799999999999 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948554+0000", 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3340000000000002e-06 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948559+0000", 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0560000000000004e-06 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948564+0000", 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1119999999999998e-06 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993455+0000", 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044891065000000001 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993493+0000", 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8321000000000001e-05 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993547+0000", 2024-04-23T23:51:52.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4038e-05 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44637 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.729338153, 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.045064633, 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44637 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948577+0000", 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948769+0000", 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019175999999999999 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948773+0000", 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9049999999999999e-06 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948782+0000", 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8740000000000001e-06 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948800+0000", 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8297000000000001e-05 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993560+0000", 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044759871999999999 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993595+0000", 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5345999999999997e-05 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993642+0000", 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6579000000000002e-05 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44638 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 2080768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.744 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.729100818999996, 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044949480999999999, 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44638 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:52.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.948814+0000", 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949028+0000", 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000213826 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949032+0000", 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949041+0000", 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1529999999999997e-06 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949045+0000", 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6979999999999997e-06 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993670+0000", 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044624979000000002 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993711+0000", 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0284999999999998e-05 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993764+0000", 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3350999999999998e-05 2024-04-23T23:51:52.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:44639 3.14 3:2edcbd7b:::rbd_data.11796b6600d0.0000000000000005:15 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.728860924000003, 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044820647999999998, 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:52.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 44639 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949054+0000", 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949329+0000", 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027439400000000002 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949334+0000", 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0819999999999998e-06 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949343+0000", 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2e-06 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.949350+0000", 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7449999999999998e-06 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993791+0000", 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.044440833999999999 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993830+0000", 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8906999999999999e-05 2024-04-23T23:51:52.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:24.993875+0000", 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5485999999999998e-05 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:52.753 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:52.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-23T23:51:52.859 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:52.859 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:52.859 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:52.859 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:52.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-23T23:51:52.941 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.1. 2024-04-23T23:51:52.950 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-04-23T23:51:52.960 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-04-23T23:51:52.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-23T23:51:52.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-23T23:51:52.972 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:52.972 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:52.972 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:52.972 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:52.972 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:52.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-23T23:51:53.077 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2024-04-23T23:51:53.077 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T23:51:53.084 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55892 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3522560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.828033399000006, 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088235701, 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55892 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269978+0000", 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3599999999999999e-07 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269980+0000", 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1430000000000001e-06 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269987+0000", 2024-04-23T23:51:53.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9369999999999998e-06 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269992+0000", 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7299999999999996e-06 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357561+0000", 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087568940999999997 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.357649+0000", 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8456999999999997e-05 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358213+0000", 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056405699999999997 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55893 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3596288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.828012518999998, 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088564025000000005, 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55893 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.269999+0000", 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270910+0000", 2024-04-23T23:51:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00091109399999999999 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270915+0000", 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0420000000000002e-06 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270928+0000", 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2788e-05 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.270935+0000", 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6129999999999998e-06 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358252+0000", 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087316478000000003 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358341+0000", 2024-04-23T23:51:53.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9143999999999999e-05 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.358563+0000", 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186600000000001 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55909 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2424832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.822154733000005, 2024-04-23T23:51:53.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088393260000000001, 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55909 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275856+0000", 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275857+0000", 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4799999999999999e-07 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275861+0000", 2024-04-23T23:51:53.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1470000000000003e-06 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275864+0000", 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2090000000000001e-06 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275867+0000", 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8650000000000001e-06 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.363938+0000", 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088070704 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364023+0000", 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5612000000000006e-05 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364250+0000", 2024-04-23T23:51:53.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022637499999999999 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55910 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2498560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.822140485999995, 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088725021000000001, 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.099 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55910 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.275871+0000", 2024-04-23T23:51:53.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277831+0000", 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001960173 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277834+0000", 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6519999999999999e-06 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277843+0000", 2024-04-23T23:51:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.551e-06 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277848+0000", 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2870000000000004e-06 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364288+0000", 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086439964999999994 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364379+0000", 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0297000000000006e-05 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.364596+0000", 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021709600000000001 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55915 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.820106553000002, 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088586247000000007, 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55915 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:53.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277905+0000", 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.36e-07 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277906+0000", 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0550000000000001e-06 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277909+0000", 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0259999999999999e-06 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277911+0000", 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5849999999999998e-06 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366139+0000", 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088227213999999998 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366223+0000", 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4639000000000001e-05 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366491+0000", 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026749199999999998 2024-04-23T23:51:53.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55916 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.820094611000002, 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088870093999999997, 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55916 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.277916+0000", 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279828+0000", 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001911149 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279831+0000", 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1949999999999999e-06 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279837+0000", 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.699e-06 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279841+0000", 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7260000000000002e-06 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366524+0000", 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086682405000000004 2024-04-23T23:51:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366575+0000", 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1020000000000003e-05 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.366787+0000", 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0002119 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55921 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3309568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.818117477000001, 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088501514000000003, 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55921 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279894+0000", 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279900+0000", 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2380000000000002e-06 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279910+0000", 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8060000000000006e-06 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279917+0000", 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0160000000000003e-06 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368070+0000", 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088153466 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368158+0000", 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7767000000000002e-05 2024-04-23T23:51:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368395+0000", 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237011 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55922 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3383296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.818089392999994, 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088734313999999995, 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55922 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279922+0000", 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8900000000000001e-07 2024-04-23T23:51:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279923+0000", 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2130000000000001e-06 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279927+0000", 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7620000000000001e-06 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279931+0000", 2024-04-23T23:51:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.72e-06 2024-04-23T23:51:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368410+0000", 2024-04-23T23:51:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088479287000000004 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368457+0000", 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7438999999999998e-05 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368656+0000", 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019860400000000001 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55923 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3457024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.818076857999998, 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089039960000000001, 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55923 2024-04-23T23:51:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.279934+0000", 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281826+0000", 2024-04-23T23:51:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00189212 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281829+0000", 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3599999999999999e-06 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281835+0000", 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8089999999999998e-06 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281838+0000", 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5109999999999999e-06 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368689+0000", 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086851337000000001 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368770+0000", 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0531999999999994e-05 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.368974+0000", 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000204291 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55927 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3751936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.816135770000002, 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088268954999999996, 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55927 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281875+0000", 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281876+0000", 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1e-07 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281877+0000", 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.95e-07 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281880+0000", 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7749999999999999e-06 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281884+0000", 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4000000000000001e-06 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369866+0000", 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.087982204999999994 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.369945+0000", 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9344000000000005e-05 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370144+0000", 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019902599999999999 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55928 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3825664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.816124423999995, 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088568117000000002, 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55928 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281887+0000", 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2700000000000001e-07 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281888+0000", 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.043e-06 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281892+0000", 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8299999999999998e-06 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281894+0000", 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6450000000000001e-06 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370172+0000", 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088277844999999994 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370217+0000", 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4388999999999998e-05 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370455+0000", 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023813799999999999 2024-04-23T23:51:53.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55929 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3899392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.816113569999999, 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088896698999999996, 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55929 2024-04-23T23:51:53.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281898+0000", 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2100000000000001e-07 2024-04-23T23:51:53.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281899+0000", 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-06 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281903+0000", 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8120000000000001e-06 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281905+0000", 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6359999999999998e-06 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370485+0000", 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088579309999999994 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370566+0000", 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1063999999999995e-05 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370794+0000", 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022863500000000001 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55930 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.816102279000006, 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089106836999999994, 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55930 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.281909+0000", 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283828+0000", 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019195690000000001 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283833+0000", 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283841+0000", 2024-04-23T23:51:53.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7499999999999992e-06 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283847+0000", 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370809+0000", 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.086961286999999998 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.370864+0000", 2024-04-23T23:51:53.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5515e-05 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371016+0000", 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015153000000000001 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55934 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4268032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.814063505999997, 2024-04-23T23:51:53.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088248662000000005, 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55934 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283948+0000", 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-07 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283950+0000", 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.59e-06 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283955+0000", 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283958+0000", 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8750000000000002e-06 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.371999+0000", 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088040765000000007 2024-04-23T23:51:53.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372043+0000", 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4202000000000001e-05 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372196+0000", 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015280200000000001 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.148 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55935 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4341760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.814047522999999, 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088588764, 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55935 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283964+0000", 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2000000000000001e-07 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283965+0000", 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5859999999999999e-06 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283973+0000", 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7419999999999997e-06 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283978+0000", 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9980000000000003e-06 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372228+0000", 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088249922999999994 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372308+0000", 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9944000000000006e-05 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372552+0000", 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024425100000000002 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.152 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55936 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4415488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.814026904000002, 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088829099999999994, 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55936 2024-04-23T23:51:53.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283984+0000", 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283985+0000", 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3200000000000001e-07 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283986+0000", 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6959999999999999e-06 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283992+0000", 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2530000000000001e-06 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.283998+0000", 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1000000000000003e-06 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372590+0000", 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088592436999999996 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372642+0000", 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2133999999999999e-05 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372813+0000", 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017114800000000001 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55937 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4489216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.814006906000003, 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089130005999999998, 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55937 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284004+0000", 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284005+0000", 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284006+0000", 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7060000000000001e-06 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284013+0000", 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2580000000000001e-06 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284017+0000", 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000003e-06 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372848+0000", 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088831630999999994 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.372928+0000", 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9548999999999993e-05 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373134+0000", 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206314 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55938 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4562944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.813988625999997, 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089404925999999996, 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55938 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.284022+0000", 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285003+0000", 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098092899999999996 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285006+0000", 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5890000000000001e-06 2024-04-23T23:51:53.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285011+0000", 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4500000000000003e-06 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285017+0000", 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373164+0000", 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088146751999999995 2024-04-23T23:51:53.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373214+0000", 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9923999999999998e-05 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373427+0000", 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021336099999999999 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.165 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55939 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4636672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.812986428000002, 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088793796999999994, 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55939 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285025+0000", 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285026+0000", 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3289999999999999e-06 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285030+0000", 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8130000000000002e-06 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285033+0000", 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7420000000000002e-06 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373457+0000", 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088423761000000003 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373537+0000", 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0167999999999995e-05 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373818+0000", 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028158700000000003 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:55940 3.16 3:68d7c3c1:::rbd_data.1179cce21f00.0000000000000000:17 [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 83.812974929000006, 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.089003014000000005, 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 55940 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.170 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285036+0000", 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2000000000000001e-07 2024-04-23T23:51:53.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285037+0000", 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0410000000000001e-06 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285041+0000", 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0920000000000002e-06 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.285043+0000", 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.689e-06 2024-04-23T23:51:53.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373841+0000", 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.088797960999999995 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.373886+0000", 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4804e-05 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:29.374039+0000", 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015320700000000001 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.173 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.174 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.174 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:53.174 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2024-04-23T23:51:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T23:51:53.175 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2024-04-23T23:51:53.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-23T23:51:53.177 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T23:51:53.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T23:51:53.279 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.280 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:53.280 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:53.280 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:53.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-23T23:51:53.391 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.391 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:53.391 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:53.391 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:53.391 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:53.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-23T23:51:53.503 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.503 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48821 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 837632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.997767976000006, 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050609529, 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.504 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48821 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519116+0000", 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4400000000000001e-07 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519118+0000", 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2479999999999999e-06 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519127+0000", 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0469999999999994e-06 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519132+0000", 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9059999999999999e-06 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569546+0000", 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050413131 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569630+0000", 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3991000000000003e-05 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569725+0000", 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5761999999999996e-05 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48822 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1015808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.997734928, 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050810847999999999, 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48822 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519149+0000", 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1200000000000003e-07 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519152+0000", 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5940000000000001e-06 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519159+0000", 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9829999999999999e-06 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519164+0000", 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5350000000000004e-06 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569757+0000", 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050592545000000003 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569846+0000", 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9127000000000002e-05 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569960+0000", 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113552 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48823 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.512 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.997712429000003, 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050991485000000003, 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48823 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.519171+0000", 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520840+0000", 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001669081 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520846+0000", 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5400000000000003e-06 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520858+0000", 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1905999999999999e-05 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.520865+0000", 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3849999999999996e-06 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.569992+0000", 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049127224999999997 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570069+0000", 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6297999999999994e-05 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.570163+0000", 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4049999999999996e-05 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48831 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.995842511999996, 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050693696000000003, 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48831 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.517 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521041+0000", 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521042+0000", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5600000000000001e-07 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521044+0000", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9139999999999998e-06 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521050+0000", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1290000000000001e-06 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521055+0000", 2024-04-23T23:51:53.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.074e-06 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571572+0000", 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050517650999999997 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571645+0000", 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2983000000000002e-05 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571735+0000", 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9488999999999996e-05 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48832 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 858112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.995814035999999, 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050893925, 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48832 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521070+0000", 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521072+0000", 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.17e-06 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521081+0000", 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5529999999999993e-06 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521086+0000", 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7779999999999998e-06 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571765+0000", 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050679645000000002 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571872+0000", 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010681899999999999 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571964+0000", 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1494000000000002e-05 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48833 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1040384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.995791592000003, 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051051645999999999, 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48833 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.521092+0000", 2024-04-23T23:51:53.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522829+0000", 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001736893 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522834+0000", 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.972e-06 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522846+0000", 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1926999999999999e-05 2024-04-23T23:51:53.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.522853+0000", 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5179999999999998e-06 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.571995+0000", 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141480000000001 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572068+0000", 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3375999999999997e-05 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.572144+0000", 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5480000000000002e-05 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48845 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.993835864000005, 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050636352000000003, 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.529 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48845 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523048+0000", 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7500000000000001e-07 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523051+0000", 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.294e-06 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523056+0000", 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9519999999999999e-06 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523061+0000", 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2289999999999997e-06 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573544+0000", 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050483541 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573592+0000", 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8196999999999999e-05 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573684+0000", 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1763999999999998e-05 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48848 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.533 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.993816737000003, 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050820054000000003, 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48848 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523067+0000", 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.84e-07 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523069+0000", 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.046e-06 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523075+0000", 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6049999999999997e-06 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523086+0000", 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1415e-05 2024-04-23T23:51:53.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573714+0000", 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050627270000000002 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573792+0000", 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8155e-05 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573887+0000", 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5179000000000005e-05 2024-04-23T23:51:53.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48851 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.993790589, 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050915421000000002, 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48851 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523093+0000", 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523095+0000", 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.483e-06 2024-04-23T23:51:53.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523097+0000", 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3700000000000002e-06 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523103+0000", 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1519999999999997e-06 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523108+0000", 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3939999999999998e-06 2024-04-23T23:51:53.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573916+0000", 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050808508000000002 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.573959+0000", 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3035999999999999e-05 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574009+0000", 2024-04-23T23:51:53.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9478000000000003e-05 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48853 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.993769896999993, 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083252000000003, 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48853 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523114+0000", 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1300000000000001e-07 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523116+0000", 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.075e-06 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523124+0000", 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7249999999999997e-06 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523129+0000", 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8640000000000004e-06 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574039+0000", 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909782000000001 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574113+0000", 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4127000000000004e-05 2024-04-23T23:51:53.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574197+0000", 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4265999999999996e-05 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48856 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 903168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.993748170999993, 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051236533000000001, 2024-04-23T23:51:53.547 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48856 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.523136+0000", 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524829+0000", 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001693586 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524834+0000", 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0050000000000004e-06 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524846+0000", 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1970000000000001e-05 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.524853+0000", 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3549999999999999e-06 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574225+0000", 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049371576 2024-04-23T23:51:53.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574291+0000", 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6087999999999995e-05 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.574372+0000", 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0952999999999996e-05 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48874 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.991851280999995, 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050726040999999999, 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48874 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525032+0000", 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525033+0000", 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0999999999999999e-07 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525035+0000", 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4509999999999999e-06 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525044+0000", 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6309999999999992e-06 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525050+0000", 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1840000000000004e-06 2024-04-23T23:51:53.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575600+0000", 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050549663000000002 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575668+0000", 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7706999999999994e-05 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575758+0000", 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0895000000000003e-05 2024-04-23T23:51:53.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48876 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 935936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.991826833999994, 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050900423, 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.556 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48876 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525057+0000", 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3500000000000002e-07 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525059+0000", 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5200000000000001e-06 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525064+0000", 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525069+0000", 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7890000000000002e-06 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575786+0000", 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050717050999999999 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575863+0000", 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6913e-05 2024-04-23T23:51:53.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575957+0000", 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4276000000000003e-05 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48879 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 940032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.991809739999994, 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051071716000000003, 2024-04-23T23:51:53.560 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48879 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.525074+0000", 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526829+0000", 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001755497 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526835+0000", 2024-04-23T23:51:53.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0810000000000001e-06 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526847+0000", 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2369e-05 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.526854+0000", 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2980000000000003e-06 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.575986+0000", 2024-04-23T23:51:53.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049131838999999997 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576055+0000", 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9288000000000005e-05 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.576146+0000", 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0344000000000001e-05 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48898 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.989832359000005, 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050731418, 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48898 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527051+0000", 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527052+0000", 2024-04-23T23:51:53.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9799999999999999e-07 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527054+0000", 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9190000000000002e-06 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527060+0000", 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1199999999999999e-06 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527064+0000", 2024-04-23T23:51:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6809999999999997e-06 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577631+0000", 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050566367000000001 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577698+0000", 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7280999999999995e-05 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577783+0000", 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4652000000000002e-05 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48906 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 980992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.989813480999999, 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050909812999999998, 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48906 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.527070+0000", 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528828+0000", 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017578419999999999 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528833+0000", 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8690000000000003e-06 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528845+0000", 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1609e-05 2024-04-23T23:51:53.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.528854+0000", 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9199999999999999e-06 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577813+0000", 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048958343000000001 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577891+0000", 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7812999999999995e-05 2024-04-23T23:51:53.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.577980+0000", 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9417000000000001e-05 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48957 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.987832440999995, 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050599264999999997, 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48957 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529051+0000", 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529052+0000", 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6699999999999999e-07 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529060+0000", 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9959999999999993e-06 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529066+0000", 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4540000000000004e-06 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529071+0000", 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579494+0000", 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050422620000000001 2024-04-23T23:51:53.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579563+0000", 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9408999999999999e-05 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579651+0000", 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7557000000000002e-05 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.577 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48958 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.987792639000006, 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050750333000000002, 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48958 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529091+0000", 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529092+0000", 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8699999999999995e-07 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529094+0000", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5289999999999998e-06 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529101+0000", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9460000000000001e-06 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529106+0000", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7639999999999996e-06 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579681+0000", 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050574846999999999 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579751+0000", 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0197999999999994e-05 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579841+0000", 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0462000000000002e-05 2024-04-23T23:51:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48959 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1026048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.987770615000002, 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050917689000000002, 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48959 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529113+0000", 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529114+0000", 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5199999999999997e-07 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529116+0000", 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3039999999999999e-06 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529123+0000", 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4089999999999997e-06 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529128+0000", 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8439999999999997e-06 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579871+0000", 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050742284999999998 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.579943+0000", 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2775999999999996e-05 2024-04-23T23:51:53.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580031+0000", 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7519e-05 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:48960 3.10 3:0b5849b1:::rbd_data.11796b6600d0.0000000000000001:15 [read 1030144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 87.987749074000007, 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.051083041000000003, 2024-04-23T23:51:53.586 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 48960 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.529135+0000", 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530836+0000", 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001701681 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530841+0000", 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0679999999999996e-06 2024-04-23T23:51:53.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530858+0000", 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6756e-05 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.530867+0000", 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3720000000000005e-06 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580061+0000", 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049194277000000002 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580129+0000", 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8647999999999997e-05 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:25.580218+0000", 2024-04-23T23:51:53.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8239000000000007e-05 2024-04-23T23:51:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.591 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:53.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-23T23:51:53.693 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:53.693 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-04-23T23:51:53.693 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:53.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-23T23:51:53.804 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.804 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-04-23T23:51:53.804 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-04-23T23:51:53.805 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-04-23T23:51:53.805 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:53.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 136.72923894300001, 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18636087200000001, 2024-04-23T23:51:53.917 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4204", 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/732867473", 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200736+0000", 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200738+0000", 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3689999999999999e-06 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200745+0000", 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4039999999999998e-06 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200757+0000", 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1816e-05 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200767+0000", 2024-04-23T23:51:53.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.011e-05 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200837+0000", 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0221000000000004e-05 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.200850+0000", 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2661e-05 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385678+0000", 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.18482810999999999 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.385910+0000", 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023178900000000001 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 0", 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386132+0000", 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022186100000000001 2024-04-23T23:51:53.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.386877+0000", 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00074578299999999997 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387026+0000", 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014845599999999999 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387068+0000", 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2009999999999999e-05 2024-04-23T23:51:53.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:49:37.387097+0000", 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9281999999999998e-05 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14641 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.380502275, 2024-04-23T23:51:53.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072488562000000006, 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14641 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:53.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549473+0000", 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5400000000000002e-07 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549475+0000", 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8020000000000001e-06 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549481+0000", 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3489999999999999e-06 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549485+0000", 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2540000000000003e-06 2024-04-23T23:51:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621850+0000", 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072365246999999994 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621901+0000", 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1091000000000003e-05 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.621962+0000", 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0464999999999998e-05 2024-04-23T23:51:53.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14642 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10977280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.380485822, 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072691657000000007, 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14642 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549489+0000", 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549490+0000", 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2099999999999997e-07 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549500+0000", 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8689999999999999e-06 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549505+0000", 2024-04-23T23:51:53.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5480000000000001e-06 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549511+0000", 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1229999999999999e-06 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622004+0000", 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072493466000000006 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622089+0000", 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5007999999999997e-05 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622181+0000", 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2121999999999997e-05 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14643 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.933 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.380459081, 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072816196999999999, 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14643 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549516+0000", 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549517+0000", 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1800000000000001e-07 2024-04-23T23:51:53.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549519+0000", 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9130000000000001e-06 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549524+0000", 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7060000000000001e-06 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549529+0000", 2024-04-23T23:51:53.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6129999999999997e-06 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622213+0000", 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072684304000000005 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622261+0000", 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7338e-05 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622332+0000", 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1904999999999997e-05 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14644 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11567104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.380440286, 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072999852000000004, 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14644 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549535+0000", 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6600000000000002e-07 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549538+0000", 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.137e-06 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549543+0000", 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6359999999999999e-06 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549548+0000", 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4750000000000004e-06 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622367+0000", 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072818916999999997 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622445+0000", 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8168000000000001e-05 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622535+0000", 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0053e-05 2024-04-23T23:51:53.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14645 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11730944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.380422702, 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073178938999999998, 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14645 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549553+0000", 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2800000000000003e-07 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549555+0000", 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6220000000000001e-06 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549566+0000", 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1226999999999999e-05 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.549576+0000", 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9420000000000007e-06 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622565+0000", 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072989233000000001 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622643+0000", 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7857999999999999e-05 2024-04-23T23:51:53.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.622732+0000", 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8728999999999997e-05 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14885 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.368727198, 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072609222000000001, 2024-04-23T23:51:53.947 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14885 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561248+0000", 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561379+0000", 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130677 2024-04-23T23:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561384+0000", 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3149999999999999e-06 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561398+0000", 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4357e-05 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561407+0000", 2024-04-23T23:51:53.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3829999999999992e-06 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633672+0000", 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072264844999999994 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633749+0000", 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7010999999999998e-05 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633857+0000", 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108634 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14886 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8241152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.36855151100001, 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571846999999995, 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14886 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561424+0000", 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561546+0000", 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012216000000000001 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561550+0000", 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6799999999999999e-06 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561558+0000", 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0390000000000005e-06 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561564+0000", 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633888+0000", 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072323080999999997 2024-04-23T23:51:53.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633946+0000", 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.838e-05 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.633996+0000", 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9734999999999997e-05 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14887 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8667136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.36839999199999, 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072633990999999995, 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14887 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561575+0000", 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561818+0000", 2024-04-23T23:51:53.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024251000000000001 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561828+0000", 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0441999999999999e-05 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561837+0000", 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1390000000000001e-06 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561844+0000", 2024-04-23T23:51:53.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4289999999999998e-06 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634030+0000", 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072186295999999997 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634121+0000", 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0977999999999995e-05 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634209+0000", 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8196999999999996e-05 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14888 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8830976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.36811736600001, 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072505643999999994, 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14888 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561858+0000", 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561974+0000", 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116406 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561978+0000", 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8349999999999997e-06 2024-04-23T23:51:53.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561987+0000", 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8650000000000007e-06 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.561995+0000", 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1599999999999998e-06 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634245+0000", 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072250152999999998 2024-04-23T23:51:53.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634287+0000", 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1931000000000002e-05 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634364+0000", 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6293999999999998e-05 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14889 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 8994816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.367971501, 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072571053999999996, 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14889 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562004+0000", 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562833+0000", 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082886600000000004 2024-04-23T23:51:53.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562837+0000", 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6450000000000004e-06 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562851+0000", 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3461e-05 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562859+0000", 2024-04-23T23:51:53.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0229999999999992e-06 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634404+0000", 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071545631999999998 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634485+0000", 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0758000000000002e-05 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.634575+0000", 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9668999999999998e-05 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14893 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 9912320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.36702181699999, 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072477084999999997, 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14893 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562953+0000", 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562954+0000", 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.82e-07 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562956+0000", 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3319999999999999e-06 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562964+0000", 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0190000000000006e-06 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562969+0000", 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0289999999999996e-06 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635255+0000", 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072285855999999996 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635339+0000", 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3418000000000007e-05 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635431+0000", 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1948999999999997e-05 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:14894 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 10076160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.366998923, 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072657008999999995, 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 14894 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.562976+0000", 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564831+0000", 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018546420000000001 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564836+0000", 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0830000000000003e-06 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564849+0000", 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2781e-05 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.564856+0000", 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5379999999999997e-06 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635461+0000", 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070604795999999997 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635534+0000", 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3249000000000003e-05 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.635633+0000", 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8919999999999995e-05 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15158 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11436032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.350983548, 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072569867999999996, 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.979 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15158 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578992+0000", 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0900000000000001e-07 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578993+0000", 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0139999999999999e-06 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578996+0000", 2024-04-23T23:51:53.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8380000000000002e-06 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.578999+0000", 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7630000000000001e-06 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651387+0000", 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072388157999999994 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651461+0000", 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4450999999999996e-05 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651562+0000", 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100435 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15159 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11599872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.35097391399999, 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072750808, 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15159 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.579001+0000", 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580830+0000", 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001828912 2024-04-23T23:51:53.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580835+0000", 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1120000000000004e-06 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580847+0000", 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.580854+0000", 2024-04-23T23:51:53.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1689999999999997e-06 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651593+0000", 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.070739390999999999 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651654+0000", 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1196000000000002e-05 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.651752+0000", 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8055999999999998e-05 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15415 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11288576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.335941553, 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072448936000000005, 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15415 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594034+0000", 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7099999999999997e-07 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594036+0000", 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8950000000000001e-06 2024-04-23T23:51:53.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594042+0000", 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594048+0000", 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666319+0000", 2024-04-23T23:51:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072271382999999995 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666395+0000", 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5622000000000002e-05 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666483+0000", 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7995e-05 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15416 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11452416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.335920229, 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072819375000000006, 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15416 2024-04-23T23:51:53.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594055+0000", 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594056+0000", 2024-04-23T23:51:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4299999999999998e-07 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594058+0000", 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0480000000000001e-06 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594065+0000", 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594070+0000", 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7690000000000002e-06 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666527+0000", 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072456522999999995 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666673+0000", 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014623699999999999 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666874+0000", 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000201196 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15417 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11616256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.33589740399999, 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073062294, 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:53.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15417 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594078+0000", 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7599999999999998e-07 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594080+0000", 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0849999999999999e-06 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:53.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594087+0000", 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0670000000000003e-06 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594093+0000", 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2769999999999996e-06 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.666956+0000", 2024-04-23T23:51:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.072863105999999997 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667043+0000", 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6773000000000004e-05 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667140+0000", 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7609999999999995e-05 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15418 3.d 3:b49b9c35:::rbd_data.117916e6274d.0000000000000002:c [read 11780096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.335874725, 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.073292189999999993, 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15418 2024-04-23T23:51:54.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.594101+0000", 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:54.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595830+0000", 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017299080000000001 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595838+0000", 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9209999999999992e-06 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595849+0000", 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0818e-05 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.595856+0000", 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3199999999999996e-06 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667199+0000", 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.071343894000000005 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667289+0000", 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9709000000000005e-05 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667393+0000", 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010362 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4473.0:15500 3.15 3:af2551ad:::rbd_data.1179129e5779.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 106.33357300199999, 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.085602783000000002, 2024-04-23T23:51:54.006 INFO:teuthology.orchestra.run.smithi119.stdout: "continuous": false, 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4473", 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 15500 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596402+0000", 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.596403+0000", 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1200000000000001e-07 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600893+0000", 2024-04-23T23:51:54.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044900160000000003 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600906+0000", 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3101999999999999e-05 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.600914+0000", 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8949999999999998e-06 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667177+0000", 2024-04-23T23:51:54.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.066262285000000004 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667292+0000", 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115338 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.667679+0000", 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038755700000000002 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.678557+0000", 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010877621000000001 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681932+0000", 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003375322 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.681977+0000", 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4650000000000001e-05 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-23T23:51:54.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-04-23T23:50:07.682005+0000", 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8085e-05 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-23T23:51:54.012 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-23T23:51:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-04-23T23:51:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T23:51:54.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-23T23:51:54.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-23T23:51:54.162 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:54.162 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:54.163 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-23T23:51:54.163 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:54.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-23T23:51:54.276 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:54.276 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-23T23:51:54.276 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-23T23:51:54.276 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-23T23:51:54.276 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:54.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-23T23:51:54.391 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:51:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-23T23:51:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-23T23:51:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-23T23:51:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61235 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.289644232000001, 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83653829499999999, 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61235 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114817+0000", 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114819+0000", 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114827+0000", 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4100000000000002e-06 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114841+0000", 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4613999999999999e-05 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114852+0000", 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1097999999999999e-05 2024-04-23T23:51:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114956+0000", 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103285 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.114971+0000", 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127904+0000", 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012932313000000001 2024-04-23T23:51:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127913+0000", 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.679e-06 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951343+0000", 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.823429195 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951356+0000", 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-04-23T23:51:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:61236 3.1e 3.db4bb21e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.287049901000003, 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83407345899999996, 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61236 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117412+0000", 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117413+0000", 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5370000000000001e-06 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117419+0000", 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117430+0000", 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553e-05 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117440+0000", 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0034000000000001e-05 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117566+0000", 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012558599999999999 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.117572+0000", 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0379999999999996e-06 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127937+0000", 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365195000000001 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.127943+0000", 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951386+0000", 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82344328700000002 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:32.951485+0000", 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9067000000000005e-05 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62505 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175394462, 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074820892, 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62505 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229067+0000", 2024-04-23T23:51:54.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229068+0000", 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229069+0000", 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229075+0000", 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7209999999999999e-06 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229080+0000", 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.339e-06 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303541+0000", 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074460994000000003 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303617+0000", 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5698000000000006e-05 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303888+0000", 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027106199999999997 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62506 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175376021999995, 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075206509000000005, 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62506 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229086+0000", 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229088+0000", 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4250000000000001e-06 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229094+0000", 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229099+0000", 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.303934+0000", 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074834932000000007 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304027+0000", 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3146999999999994e-05 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304292+0000", 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026521999999999998 2024-04-23T23:51:54.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62509 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175356915999998, 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615238000000001, 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62509 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229105+0000", 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229107+0000", 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229112+0000", 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229118+0000", 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.417e-06 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304327+0000", 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075209814999999999 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304407+0000", 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9319000000000004e-05 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304720+0000", 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313226 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62510 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175338406999998, 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075988704000000004, 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62510 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229123+0000", 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229124+0000", 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229126+0000", 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069999999999999e-06 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229132+0000", 2024-04-23T23:51:54.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229136+0000", 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4490000000000001e-06 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304752+0000", 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075615748999999996 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.304842+0000", 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9456999999999994e-05 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305112+0000", 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027026600000000002 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62518 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175235357999995, 2024-04-23T23:51:54.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076272696000000001, 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62518 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229226+0000", 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229227+0000", 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229229+0000", 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-23T23:51:54.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229235+0000", 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0090000000000004e-06 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229240+0000", 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305157+0000", 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075917581999999997 2024-04-23T23:51:54.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305237+0000", 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0117999999999994e-05 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305499+0000", 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026162499999999998 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62519 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175215738999995, 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076609143000000005, 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62519 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229246+0000", 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229269+0000", 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2677999999999999e-05 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229334+0000", 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-05 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229341+0000", 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2024-04-23T23:51:54.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305546+0000", 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076205145000000002 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305642+0000", 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5884000000000006e-05 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305855+0000", 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000213465 2024-04-23T23:51:54.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62522 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175110263999997, 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076818779000000004, 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62522 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229351+0000", 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229352+0000", 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-07 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229355+0000", 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229363+0000", 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6070000000000002e-06 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229369+0000", 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305886+0000", 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076517140999999997 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.305946+0000", 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9451999999999999e-05 2024-04-23T23:51:54.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306170+0000", 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022438100000000001 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62523 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175085078999999, 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077158305999999996, 2024-04-23T23:51:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62523 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229377+0000", 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229379+0000", 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229386+0000", 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229401+0000", 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5434000000000002e-05 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306201+0000", 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.076799695000000001 2024-04-23T23:51:54.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306301+0000", 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100103 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306535+0000", 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234007 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62526 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175051534000005, 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077750112999999996, 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62526 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229410+0000", 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229411+0000", 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229414+0000", 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3459999999999998e-06 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229421+0000", 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229427+0000", 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306572+0000", 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077144721999999999 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.306848+0000", 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000276173 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307160+0000", 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031240800000000002 2024-04-23T23:51:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62529 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175027771000003, 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078645458000000001, 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62529 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229434+0000", 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229437+0000", 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2309999999999999e-06 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229443+0000", 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229448+0000", 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307234+0000", 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077785879000000002 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.307403+0000", 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016940799999999999 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308079+0000", 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000676003 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62530 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.175006730000007, 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079338027000000005, 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62530 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229455+0000", 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229457+0000", 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229494+0000", 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7100000000000001e-05 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229514+0000", 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9364e-05 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308114+0000", 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.078600254999999994 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308200+0000", 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6083999999999998e-05 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308793+0000", 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059295500000000004 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62533 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:54.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174938495000006, 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079848246999999997, 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62533 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229523+0000", 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229524+0000", 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229528+0000", 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0409999999999999e-06 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229535+0000", 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3429999999999998e-06 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229541+0000", 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3589999999999999e-06 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308825+0000", 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079284582000000006 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.308906+0000", 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1017e-05 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309371+0000", 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046528800000000001 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62534 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174914027, 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080166256000000005, 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62534 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229548+0000", 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229550+0000", 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1909999999999999e-06 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229558+0000", 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9529999999999999e-06 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229563+0000", 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309415+0000", 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.079851697999999999 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309495+0000", 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0568000000000005e-05 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309714+0000", 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021857199999999999 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62537 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174888328999998, 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080640497000000005, 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62537 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229573+0000", 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229574+0000", 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229576+0000", 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229583+0000", 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9879999999999998e-06 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229588+0000", 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309745+0000", 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080156272000000001 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.309825+0000", 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0395999999999994e-05 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310214+0000", 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038890700000000002 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62538 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:54.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174866562999995, 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081078197000000005, 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62538 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229595+0000", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229598+0000", 2024-04-23T23:51:54.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0870000000000001e-06 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229604+0000", 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229609+0000", 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0599999999999998e-06 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310254+0000", 2024-04-23T23:51:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080644718000000004 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310369+0000", 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011537800000000001 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310673+0000", 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426699999999999 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62540 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174846097, 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081868275000000004, 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62540 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.467 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229616+0000", 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-23T23:51:54.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229618+0000", 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1119999999999999e-06 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229624+0000", 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229630+0000", 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0750000000000003e-06 2024-04-23T23:51:54.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310705+0000", 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081074489 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.310805+0000", 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010002599999999999 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311484+0000", 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067906300000000004 2024-04-23T23:51:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62542 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174768237999999, 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082298957000000006, 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.471 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62542 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229693+0000", 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229694+0000", 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229697+0000", 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0989999999999999e-06 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229707+0000", 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7149999999999993e-06 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229713+0000", 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5939999999999999e-06 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311520+0000", 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.081806527000000004 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311612+0000", 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1984000000000006e-05 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.311992+0000", 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038040099999999999 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4473.0:62543 3.b 3:d51392fd:::rbd_data.117966a158ce.000000000000000a:19 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.174738586999993, 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082571122999999996, 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-23T23:51:54.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4473", 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1891235274", 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62543 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229723+0000", 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-23T23:51:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229724+0000", 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1799999999999995e-07 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229726+0000", 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2120000000000002e-06 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229733+0000", 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7880000000000001e-06 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.229747+0000", 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4314999999999999e-05 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312024+0000", 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082276751999999995 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312102+0000", 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8258999999999997e-05 2024-04-23T23:51:54.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-23T23:50:34.312294+0000", 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019227900000000001 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-23T23:51:54.479 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:51:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-23T23:51:54.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T23:51:56.014 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T23:51:56.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-23T23:51:57.018 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-23T23:51:57.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T23:51:58.025 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T23:51:58.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-23T23:51:59.027 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-23T23:51:59.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-23T23:52:00.042 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-04-23T23:52:00.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-23T23:52:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-23T23:52:01.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-23T23:52:02.054 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-04-23T23:52:02.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-23T23:52:02.948 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-23T23:52:02.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-23T23:52:04.061 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-04-23T23:52:04.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-23T23:52:05.067 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-23T23:52:05.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-23T23:52:06.069 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-04-23T23:52:06.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-23T23:52:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-23T23:52:07.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T23:52:08.082 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T23:52:08.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-23T23:52:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-23T23:52:09.110 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-23T23:52:09.111 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-23T23:52:09.111 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-23T23:52:09.111 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-23T23:52:09.111 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T23:52:09.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T23:52:09.400 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:09.401 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":148,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:52:09.101343+0000","last_up_change":"2024-04-23T23:50:32.112122+0000","last_in_change":"2024-04-23T23:51:52.954149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T23:49:35.387984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T23:49:41.909447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-04-23T23:49:49.482932+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6820","nonce":3364976130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6824","nonce":3364976130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6831","nonce":3364976130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6828","nonce":3364976130}]},"public_addr":"172.21.15.92:6820/3364976130","cluster_addr":"172.21.15.92:6824/3364976130","heartbeat_back_addr":"172.21.15.92:6831/3364976130","heartbeat_front_addr":"172.21.15.92:6828/3364976130","state":["exists","up"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6817","nonce":2924722109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6821","nonce":2924722109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6829","nonce":2924722109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6823","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6825","nonce":2924722109}]},"public_addr":"172.21.15.92:6817/2924722109","cluster_addr":"172.21.15.92:6821/2924722109","heartbeat_back_addr":"172.21.15.92:6829/2924722109","heartbeat_front_addr":"172.21.15.92:6825/2924722109","state":["exists","up"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":38,"up_from":56,"up_thru":135,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6809","nonce":1499077312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6811","nonce":1499077312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6815","nonce":1499077312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6813","nonce":1499077312}]},"public_addr":"172.21.15.92:6809/1499077312","cluster_addr":"172.21.15.92:6811/1499077312","heartbeat_back_addr":"172.21.15.92:6815/1499077312","heartbeat_front_addr":"172.21.15.92:6813/1499077312","state":["exists","up"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.405502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.334094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.367807+0000","dead_epoch":39},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.328752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.255936+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.298943+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.352632+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[5,3]},{"pgid":"3.14","osds":[1,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:52:09.413 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T23:52:09.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T23:52:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:09.699 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":148,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:52:09.101343+0000","last_up_change":"2024-04-23T23:50:32.112122+0000","last_in_change":"2024-04-23T23:51:52.954149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T23:49:35.387984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T23:49:41.909447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-04-23T23:49:49.482932+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6820","nonce":3364976130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6824","nonce":3364976130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6831","nonce":3364976130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6828","nonce":3364976130}]},"public_addr":"172.21.15.92:6820/3364976130","cluster_addr":"172.21.15.92:6824/3364976130","heartbeat_back_addr":"172.21.15.92:6831/3364976130","heartbeat_front_addr":"172.21.15.92:6828/3364976130","state":["exists","up"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6817","nonce":2924722109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6821","nonce":2924722109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6829","nonce":2924722109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6823","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6825","nonce":2924722109}]},"public_addr":"172.21.15.92:6817/2924722109","cluster_addr":"172.21.15.92:6821/2924722109","heartbeat_back_addr":"172.21.15.92:6829/2924722109","heartbeat_front_addr":"172.21.15.92:6825/2924722109","state":["exists","up"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":38,"up_from":56,"up_thru":135,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6809","nonce":1499077312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6811","nonce":1499077312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6815","nonce":1499077312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6813","nonce":1499077312}]},"public_addr":"172.21.15.92:6809/1499077312","cluster_addr":"172.21.15.92:6811/1499077312","heartbeat_back_addr":"172.21.15.92:6815/1499077312","heartbeat_front_addr":"172.21.15.92:6813/1499077312","state":["exists","up"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.405502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.334094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.367807+0000","dead_epoch":39},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.328752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.255936+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.298943+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.352632+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[5,3]},{"pgid":"3.14","osds":[1,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:52:09.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T23:52:09.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T23:52:09.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T23:52:09.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T23:52:09.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-23T23:52:09.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-23T23:52:09.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-23T23:52:09.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-23T23:52:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:64424509475 2024-04-23T23:52:09.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T23:52:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:240518168597 2024-04-23T23:52:09.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T23:52:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:60129542179 2024-04-23T23:52:09.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T23:52:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:55834574882 2024-04-23T23:52:09.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T23:52:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:64424509476 2024-04-23T23:52:09.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T23:52:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:60129542180 2024-04-23T23:52:09.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T23:52:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:60129542179 2024-04-23T23:52:09.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T23:52:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:60129542178 2024-04-23T23:52:09.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T23:52:10.543 INFO:teuthology.orchestra.run.smithi092.stdout:64424509474 2024-04-23T23:52:10.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.0 2024-04-23T23:52:10.579 INFO:teuthology.orchestra.run.smithi092.stdout:60129542177 2024-04-23T23:52:10.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542177 for osd.7 2024-04-23T23:52:10.604 INFO:teuthology.orchestra.run.smithi092.stdout:55834574881 2024-04-23T23:52:10.605 INFO:teuthology.orchestra.run.smithi092.stdout:60129542177 2024-04-23T23:52:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:64424509475 2024-04-23T23:52:10.613 INFO:teuthology.orchestra.run.smithi092.stdout:240518168596 2024-04-23T23:52:10.615 INFO:teuthology.orchestra.run.smithi092.stdout:60129542176 2024-04-23T23:52:10.625 INFO:teuthology.orchestra.run.smithi092.stdout:60129542178 2024-04-23T23:52:10.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.2 2024-04-23T23:52:10.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542177 for osd.4 2024-04-23T23:52:10.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542178 got 60129542176 for osd.6 2024-04-23T23:52:10.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.1 2024-04-23T23:52:10.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 240518168597 got 240518168596 for osd.3 2024-04-23T23:52:10.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542178 for osd.5 2024-04-23T23:52:11.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T23:52:11.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T23:52:11.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T23:52:11.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T23:52:11.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T23:52:11.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T23:52:11.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T23:52:11.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T23:52:12.071 INFO:teuthology.orchestra.run.smithi092.stdout:64424509474 2024-04-23T23:52:12.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.0 2024-04-23T23:52:12.171 INFO:teuthology.orchestra.run.smithi092.stdout:240518168598 2024-04-23T23:52:12.180 INFO:teuthology.orchestra.run.smithi092.stdout:60129542179 2024-04-23T23:52:12.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 240518168597 got 240518168598 for osd.3 2024-04-23T23:52:12.185 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:12.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542179 for osd.7 2024-04-23T23:52:12.195 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:12.210 INFO:teuthology.orchestra.run.smithi092.stdout:55834574882 2024-04-23T23:52:12.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.2 2024-04-23T23:52:12.223 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:12.233 INFO:teuthology.orchestra.run.smithi092.stdout:60129542178 2024-04-23T23:52:12.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542178 got 60129542178 for osd.6 2024-04-23T23:52:12.247 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:12.255 INFO:teuthology.orchestra.run.smithi092.stdout:60129542179 2024-04-23T23:52:12.260 INFO:teuthology.orchestra.run.smithi092.stdout:60129542180 2024-04-23T23:52:12.266 INFO:teuthology.orchestra.run.smithi092.stdout:64424509477 2024-04-23T23:52:12.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542179 for osd.4 2024-04-23T23:52:12.269 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:12.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542180 for osd.5 2024-04-23T23:52:12.274 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:12.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509477 for osd.1 2024-04-23T23:52:12.279 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:13.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T23:52:13.381 INFO:teuthology.orchestra.run.smithi092.stdout:64424509476 2024-04-23T23:52:13.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509476 for osd.0 2024-04-23T23:52:13.395 DEBUG:teuthology.parallel:result is None 2024-04-23T23:52:13.395 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T23:52:13.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:13.676 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:13.687 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T23:52:13.383614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16686,"ondisk_log_size":16686,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389416,"kb_used_data":14896,"kb_used_omap":21,"kb_used_meta":374378,"kb_avail":754585304,"statfs":{"total":773094113280,"available":772695351296,"internally_reserved":0,"allocated":15253504,"data_stored":7922414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":383363445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.292260"},"pg_stats":[{"pgid":"3.1f","version":"131'661","reported_seq":8954,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107934+0000","last_change":"2024-04-23T23:51:51.018566+0000","last_active":"2024-04-23T23:52:09.107934+0000","last_peered":"2024-04-23T23:52:09.107934+0000","last_clean":"2024-04-23T23:52:09.107934+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:09.107934+0000","last_undegraded":"2024-04-23T23:52:09.107934+0000","last_fullsized":"2024-04-23T23:52:09.107934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:45.446677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'834","reported_seq":4329,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096720+0000","last_change":"2024-04-23T23:51:50.007789+0000","last_active":"2024-04-23T23:52:10.096720+0000","last_peered":"2024-04-23T23:52:10.096720+0000","last_clean":"2024-04-23T23:52:10.096720+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:10.096720+0000","last_undegraded":"2024-04-23T23:52:10.096720+0000","last_fullsized":"2024-04-23T23:52:10.096720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:44:24.111886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"119'379","reported_seq":3308,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382998+0000","last_change":"2024-04-23T23:51:50.014697+0000","last_active":"2024-04-23T23:52:09.382998+0000","last_peered":"2024-04-23T23:52:09.382998+0000","last_clean":"2024-04-23T23:52:09.382998+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:09.382998+0000","last_undegraded":"2024-04-23T23:52:09.382998+0000","last_fullsized":"2024-04-23T23:52:09.382998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:40.316577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"131'741","reported_seq":6498,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.383104+0000","last_change":"2024-04-23T23:51:50.013253+0000","last_active":"2024-04-23T23:52:09.383104+0000","last_peered":"2024-04-23T23:52:09.383104+0000","last_clean":"2024-04-23T23:52:09.383104+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:09.383104+0000","last_undegraded":"2024-04-23T23:52:09.383104+0000","last_fullsized":"2024-04-23T23:52:09.383104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:47.200572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"129'304","reported_seq":4086,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107808+0000","last_change":"2024-04-23T23:51:51.017335+0000","last_active":"2024-04-23T23:52:09.107808+0000","last_peered":"2024-04-23T23:52:09.107808+0000","last_clean":"2024-04-23T23:52:09.107808+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:09.107808+0000","last_undegraded":"2024-04-23T23:52:09.107808+0000","last_fullsized":"2024-04-23T23:52:09.107808+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:52:41.586532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"123'333","reported_seq":4979,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092378+0000","last_change":"2024-04-23T23:51:57.043361+0000","last_active":"2024-04-23T23:52:07.092378+0000","last_peered":"2024-04-23T23:52:07.092378+0000","last_clean":"2024-04-23T23:52:07.092378+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:07.092378+0000","last_undegraded":"2024-04-23T23:52:07.092378+0000","last_fullsized":"2024-04-23T23:52:07.092378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:06.039919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"119'788","reported_seq":6488,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107545+0000","last_change":"2024-04-23T23:51:57.044986+0000","last_active":"2024-04-23T23:52:09.107545+0000","last_peered":"2024-04-23T23:52:09.107545+0000","last_clean":"2024-04-23T23:52:09.107545+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:09.107545+0000","last_undegraded":"2024-04-23T23:52:09.107545+0000","last_fullsized":"2024-04-23T23:52:09.107545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:22.571801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"126'626","reported_seq":3564,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097250+0000","last_change":"2024-04-23T23:51:51.014869+0000","last_active":"2024-04-23T23:52:10.097250+0000","last_peered":"2024-04-23T23:52:10.097250+0000","last_clean":"2024-04-23T23:52:10.097250+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:10.097250+0000","last_undegraded":"2024-04-23T23:52:10.097250+0000","last_fullsized":"2024-04-23T23:52:10.097250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:25.995036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"131'620","reported_seq":6367,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107990+0000","last_change":"2024-04-23T23:51:51.021741+0000","last_active":"2024-04-23T23:52:09.107990+0000","last_peered":"2024-04-23T23:52:09.107990+0000","last_clean":"2024-04-23T23:52:09.107990+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:09.107990+0000","last_undegraded":"2024-04-23T23:52:09.107990+0000","last_fullsized":"2024-04-23T23:52:09.107990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:28:57.626808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"131'468","reported_seq":6523,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382726+0000","last_change":"2024-04-23T23:51:50.012988+0000","last_active":"2024-04-23T23:52:09.382726+0000","last_peered":"2024-04-23T23:52:09.382726+0000","last_clean":"2024-04-23T23:52:09.382726+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:09.382726+0000","last_undegraded":"2024-04-23T23:52:09.382726+0000","last_fullsized":"2024-04-23T23:52:09.382726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:48.845107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"129'366","reported_seq":2162,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844525+0000","last_change":"2024-04-23T23:51:50.011281+0000","last_active":"2024-04-23T23:52:07.844525+0000","last_peered":"2024-04-23T23:52:07.844525+0000","last_clean":"2024-04-23T23:52:07.844525+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:07.844525+0000","last_undegraded":"2024-04-23T23:52:07.844525+0000","last_fullsized":"2024-04-23T23:52:07.844525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:11.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"125'277","reported_seq":4560,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108088+0000","last_change":"2024-04-23T23:51:57.042203+0000","last_active":"2024-04-23T23:52:09.108088+0000","last_peered":"2024-04-23T23:52:09.108088+0000","last_clean":"2024-04-23T23:52:09.108088+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:09.108088+0000","last_undegraded":"2024-04-23T23:52:09.108088+0000","last_fullsized":"2024-04-23T23:52:09.108088+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:19.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105223+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:52:09.105223+0000","last_peered":"2024-04-23T23:52:09.105223+0000","last_clean":"2024-04-23T23:52:09.105223+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:09.105223+0000","last_undegraded":"2024-04-23T23:52:09.105223+0000","last_fullsized":"2024-04-23T23:52:09.105223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"131'864","reported_seq":3092,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107941+0000","last_change":"2024-04-23T23:51:57.042767+0000","last_active":"2024-04-23T23:52:09.107941+0000","last_peered":"2024-04-23T23:52:09.107941+0000","last_clean":"2024-04-23T23:52:09.107941+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:09.107941+0000","last_undegraded":"2024-04-23T23:52:09.107941+0000","last_fullsized":"2024-04-23T23:52:09.107941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:28:58.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107742+0000","last_change":"2024-04-23T23:51:53.964555+0000","last_active":"2024-04-23T23:52:09.107742+0000","last_peered":"2024-04-23T23:52:09.107742+0000","last_clean":"2024-04-23T23:52:09.107742+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:09.107742+0000","last_undegraded":"2024-04-23T23:52:09.107742+0000","last_fullsized":"2024-04-23T23:52:09.107742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:05:37.770013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'624","reported_seq":4639,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096820+0000","last_change":"2024-04-23T23:51:50.009306+0000","last_active":"2024-04-23T23:52:10.096820+0000","last_peered":"2024-04-23T23:52:10.096820+0000","last_clean":"2024-04-23T23:52:10.096820+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:10.096820+0000","last_undegraded":"2024-04-23T23:52:10.096820+0000","last_fullsized":"2024-04-23T23:52:10.096820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:34.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":272,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844652+0000","last_change":"2024-04-23T23:51:47.967029+0000","last_active":"2024-04-23T23:52:07.844652+0000","last_peered":"2024-04-23T23:52:07.844652+0000","last_clean":"2024-04-23T23:52:07.844652+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:07.844652+0000","last_undegraded":"2024-04-23T23:52:07.844652+0000","last_fullsized":"2024-04-23T23:52:07.844652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:28.475010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'460","reported_seq":2674,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107932+0000","last_change":"2024-04-23T23:51:57.043327+0000","last_active":"2024-04-23T23:52:09.107932+0000","last_peered":"2024-04-23T23:52:09.107932+0000","last_clean":"2024-04-23T23:52:09.107932+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:09.107932+0000","last_undegraded":"2024-04-23T23:52:09.107932+0000","last_fullsized":"2024-04-23T23:52:09.107932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:35.662689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108102+0000","last_change":"2024-04-23T23:51:53.964521+0000","last_active":"2024-04-23T23:52:09.108102+0000","last_peered":"2024-04-23T23:52:09.108102+0000","last_clean":"2024-04-23T23:52:09.108102+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:09.108102+0000","last_undegraded":"2024-04-23T23:52:09.108102+0000","last_fullsized":"2024-04-23T23:52:09.108102+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:14:20.619635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'541","reported_seq":9060,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382900+0000","last_change":"2024-04-23T23:51:50.014733+0000","last_active":"2024-04-23T23:52:09.382900+0000","last_peered":"2024-04-23T23:52:09.382900+0000","last_clean":"2024-04-23T23:52:09.382900+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:09.382900+0000","last_undegraded":"2024-04-23T23:52:09.382900+0000","last_fullsized":"2024-04-23T23:52:09.382900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:07.885750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"131'742","reported_seq":5032,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092516+0000","last_change":"2024-04-23T23:51:51.016283+0000","last_active":"2024-04-23T23:52:07.092516+0000","last_peered":"2024-04-23T23:52:07.092516+0000","last_clean":"2024-04-23T23:52:07.092516+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:07.092516+0000","last_undegraded":"2024-04-23T23:52:07.092516+0000","last_fullsized":"2024-04-23T23:52:07.092516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:19.673527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382865+0000","last_change":"2024-04-23T23:51:53.965136+0000","last_active":"2024-04-23T23:52:09.382865+0000","last_peered":"2024-04-23T23:52:09.382865+0000","last_clean":"2024-04-23T23:52:09.382865+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:09.382865+0000","last_undegraded":"2024-04-23T23:52:09.382865+0000","last_fullsized":"2024-04-23T23:52:09.382865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:40.481944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"126'317","reported_seq":2745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107507+0000","last_change":"2024-04-23T23:51:57.045408+0000","last_active":"2024-04-23T23:52:09.107507+0000","last_peered":"2024-04-23T23:52:09.107507+0000","last_clean":"2024-04-23T23:52:09.107507+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:09.107507+0000","last_undegraded":"2024-04-23T23:52:09.107507+0000","last_fullsized":"2024-04-23T23:52:09.107507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:34.567554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":271,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.547959+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:52:09.547959+0000","last_peered":"2024-04-23T23:52:09.547959+0000","last_clean":"2024-04-23T23:52:09.547959+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:09.547959+0000","last_undegraded":"2024-04-23T23:52:09.547959+0000","last_fullsized":"2024-04-23T23:52:09.547959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'353","reported_seq":3179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107819+0000","last_change":"2024-04-23T23:51:51.012712+0000","last_active":"2024-04-23T23:52:09.107819+0000","last_peered":"2024-04-23T23:52:09.107819+0000","last_clean":"2024-04-23T23:52:09.107819+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:09.107819+0000","last_undegraded":"2024-04-23T23:52:09.107819+0000","last_fullsized":"2024-04-23T23:52:09.107819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:52.621838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":282,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844466+0000","last_change":"2024-04-23T23:51:53.964436+0000","last_active":"2024-04-23T23:52:07.844466+0000","last_peered":"2024-04-23T23:52:07.844466+0000","last_clean":"2024-04-23T23:52:07.844466+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:07.844466+0000","last_undegraded":"2024-04-23T23:52:07.844466+0000","last_fullsized":"2024-04-23T23:52:07.844466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:59.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"129'630","reported_seq":2024,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096571+0000","last_change":"2024-04-23T23:51:50.007306+0000","last_active":"2024-04-23T23:52:10.096571+0000","last_peered":"2024-04-23T23:52:10.096571+0000","last_clean":"2024-04-23T23:52:10.096571+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:10.096571+0000","last_undegraded":"2024-04-23T23:52:10.096571+0000","last_fullsized":"2024-04-23T23:52:10.096571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:42.222266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382664+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:52:09.382664+0000","last_peered":"2024-04-23T23:52:09.382664+0000","last_clean":"2024-04-23T23:52:09.382664+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:09.382664+0000","last_undegraded":"2024-04-23T23:52:09.382664+0000","last_fullsized":"2024-04-23T23:52:09.382664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":444,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844531+0000","last_change":"2024-04-23T23:51:51.009383+0000","last_active":"2024-04-23T23:52:07.844531+0000","last_peered":"2024-04-23T23:52:07.844531+0000","last_clean":"2024-04-23T23:52:07.844531+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:07.844531+0000","last_undegraded":"2024-04-23T23:52:07.844531+0000","last_fullsized":"2024-04-23T23:52:07.844531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:07.935657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"125'321","reported_seq":1922,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096768+0000","last_change":"2024-04-23T23:51:57.041481+0000","last_active":"2024-04-23T23:52:10.096768+0000","last_peered":"2024-04-23T23:52:10.096768+0000","last_clean":"2024-04-23T23:52:10.096768+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:10.096768+0000","last_undegraded":"2024-04-23T23:52:10.096768+0000","last_fullsized":"2024-04-23T23:52:10.096768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:07.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'516","reported_seq":2892,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092470+0000","last_change":"2024-04-23T23:51:50.014340+0000","last_active":"2024-04-23T23:52:07.092470+0000","last_peered":"2024-04-23T23:52:07.092470+0000","last_clean":"2024-04-23T23:52:07.092470+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:07.092470+0000","last_undegraded":"2024-04-23T23:52:07.092470+0000","last_fullsized":"2024-04-23T23:52:07.092470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:49:40.738996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"129'442","reported_seq":3505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:50.008342+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:00.995303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'443","reported_seq":2952,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097048+0000","last_change":"2024-04-23T23:51:51.016535+0000","last_active":"2024-04-23T23:52:10.097048+0000","last_peered":"2024-04-23T23:52:10.097048+0000","last_clean":"2024-04-23T23:52:10.097048+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:10.097048+0000","last_undegraded":"2024-04-23T23:52:10.097048+0000","last_fullsized":"2024-04-23T23:52:10.097048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:47:29.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'443","reported_seq":4073,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382915+0000","last_change":"2024-04-23T23:51:50.014803+0000","last_active":"2024-04-23T23:52:09.382915+0000","last_peered":"2024-04-23T23:52:09.382915+0000","last_clean":"2024-04-23T23:52:09.382915+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:09.382915+0000","last_undegraded":"2024-04-23T23:52:09.382915+0000","last_fullsized":"2024-04-23T23:52:09.382915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:47.080894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'363","reported_seq":3494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844572+0000","last_change":"2024-04-23T23:51:50.012198+0000","last_active":"2024-04-23T23:52:07.844572+0000","last_peered":"2024-04-23T23:52:07.844572+0000","last_clean":"2024-04-23T23:52:07.844572+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:07.844572+0000","last_undegraded":"2024-04-23T23:52:07.844572+0000","last_fullsized":"2024-04-23T23:52:07.844572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.339718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"123'586","reported_seq":3674,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844493+0000","last_change":"2024-04-23T23:51:50.010749+0000","last_active":"2024-04-23T23:52:07.844493+0000","last_peered":"2024-04-23T23:52:07.844493+0000","last_clean":"2024-04-23T23:52:07.844493+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:07.844493+0000","last_undegraded":"2024-04-23T23:52:07.844493+0000","last_fullsized":"2024-04-23T23:52:07.844493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:23.648951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"131'858","reported_seq":4029,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844441+0000","last_change":"2024-04-23T23:51:50.010902+0000","last_active":"2024-04-23T23:52:07.844441+0000","last_peered":"2024-04-23T23:52:07.844441+0000","last_clean":"2024-04-23T23:52:07.844441+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:07.844441+0000","last_undegraded":"2024-04-23T23:52:07.844441+0000","last_fullsized":"2024-04-23T23:52:07.844441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:29.439340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"129'321","reported_seq":5844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:51.012948+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:42.912702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"118'411","reported_seq":4195,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844358+0000","last_change":"2024-04-23T23:51:50.007735+0000","last_active":"2024-04-23T23:52:07.844358+0000","last_peered":"2024-04-23T23:52:07.844358+0000","last_clean":"2024-04-23T23:52:07.844358+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:07.844358+0000","last_undegraded":"2024-04-23T23:52:07.844358+0000","last_fullsized":"2024-04-23T23:52:07.844358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:10:47.430347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"126'451","reported_seq":2347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382665+0000","last_change":"2024-04-23T23:51:50.011706+0000","last_active":"2024-04-23T23:52:09.382665+0000","last_peered":"2024-04-23T23:52:09.382665+0000","last_clean":"2024-04-23T23:52:09.382665+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:09.382665+0000","last_undegraded":"2024-04-23T23:52:09.382665+0000","last_fullsized":"2024-04-23T23:52:09.382665+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:50.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"126'486","reported_seq":3064,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844367+0000","last_change":"2024-04-23T23:51:50.011755+0000","last_active":"2024-04-23T23:52:07.844367+0000","last_peered":"2024-04-23T23:52:07.844367+0000","last_clean":"2024-04-23T23:52:07.844367+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:07.844367+0000","last_undegraded":"2024-04-23T23:52:07.844367+0000","last_fullsized":"2024-04-23T23:52:07.844367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:45:45.614364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16569,"ondisk_log_size":16569,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168598,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46924,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324916,"statfs":{"total":96636764160,"available":96588713984,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2338816,"data_stored":1427202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50936,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320904,"statfs":{"total":96636764160,"available":96584605696,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":836853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:13.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:13.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:13.982 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T23:52:13.383614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16686,"ondisk_log_size":16686,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389416,"kb_used_data":14896,"kb_used_omap":21,"kb_used_meta":374378,"kb_avail":754585304,"statfs":{"total":773094113280,"available":772695351296,"internally_reserved":0,"allocated":15253504,"data_stored":7922414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":383363445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.292260"},"pg_stats":[{"pgid":"3.1f","version":"131'661","reported_seq":8954,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107934+0000","last_change":"2024-04-23T23:51:51.018566+0000","last_active":"2024-04-23T23:52:09.107934+0000","last_peered":"2024-04-23T23:52:09.107934+0000","last_clean":"2024-04-23T23:52:09.107934+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:09.107934+0000","last_undegraded":"2024-04-23T23:52:09.107934+0000","last_fullsized":"2024-04-23T23:52:09.107934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:45.446677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'834","reported_seq":4329,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096720+0000","last_change":"2024-04-23T23:51:50.007789+0000","last_active":"2024-04-23T23:52:10.096720+0000","last_peered":"2024-04-23T23:52:10.096720+0000","last_clean":"2024-04-23T23:52:10.096720+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:10.096720+0000","last_undegraded":"2024-04-23T23:52:10.096720+0000","last_fullsized":"2024-04-23T23:52:10.096720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:44:24.111886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"119'379","reported_seq":3308,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382998+0000","last_change":"2024-04-23T23:51:50.014697+0000","last_active":"2024-04-23T23:52:09.382998+0000","last_peered":"2024-04-23T23:52:09.382998+0000","last_clean":"2024-04-23T23:52:09.382998+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:09.382998+0000","last_undegraded":"2024-04-23T23:52:09.382998+0000","last_fullsized":"2024-04-23T23:52:09.382998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:40.316577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"131'741","reported_seq":6498,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.383104+0000","last_change":"2024-04-23T23:51:50.013253+0000","last_active":"2024-04-23T23:52:09.383104+0000","last_peered":"2024-04-23T23:52:09.383104+0000","last_clean":"2024-04-23T23:52:09.383104+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:09.383104+0000","last_undegraded":"2024-04-23T23:52:09.383104+0000","last_fullsized":"2024-04-23T23:52:09.383104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:47.200572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"129'304","reported_seq":4086,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107808+0000","last_change":"2024-04-23T23:51:51.017335+0000","last_active":"2024-04-23T23:52:09.107808+0000","last_peered":"2024-04-23T23:52:09.107808+0000","last_clean":"2024-04-23T23:52:09.107808+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:09.107808+0000","last_undegraded":"2024-04-23T23:52:09.107808+0000","last_fullsized":"2024-04-23T23:52:09.107808+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:52:41.586532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"123'333","reported_seq":4979,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092378+0000","last_change":"2024-04-23T23:51:57.043361+0000","last_active":"2024-04-23T23:52:07.092378+0000","last_peered":"2024-04-23T23:52:07.092378+0000","last_clean":"2024-04-23T23:52:07.092378+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:07.092378+0000","last_undegraded":"2024-04-23T23:52:07.092378+0000","last_fullsized":"2024-04-23T23:52:07.092378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:06.039919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"119'788","reported_seq":6488,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107545+0000","last_change":"2024-04-23T23:51:57.044986+0000","last_active":"2024-04-23T23:52:09.107545+0000","last_peered":"2024-04-23T23:52:09.107545+0000","last_clean":"2024-04-23T23:52:09.107545+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:09.107545+0000","last_undegraded":"2024-04-23T23:52:09.107545+0000","last_fullsized":"2024-04-23T23:52:09.107545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:22.571801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"126'626","reported_seq":3564,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097250+0000","last_change":"2024-04-23T23:51:51.014869+0000","last_active":"2024-04-23T23:52:10.097250+0000","last_peered":"2024-04-23T23:52:10.097250+0000","last_clean":"2024-04-23T23:52:10.097250+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:10.097250+0000","last_undegraded":"2024-04-23T23:52:10.097250+0000","last_fullsized":"2024-04-23T23:52:10.097250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:25.995036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"131'620","reported_seq":6367,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107990+0000","last_change":"2024-04-23T23:51:51.021741+0000","last_active":"2024-04-23T23:52:09.107990+0000","last_peered":"2024-04-23T23:52:09.107990+0000","last_clean":"2024-04-23T23:52:09.107990+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:09.107990+0000","last_undegraded":"2024-04-23T23:52:09.107990+0000","last_fullsized":"2024-04-23T23:52:09.107990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:28:57.626808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"131'468","reported_seq":6523,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382726+0000","last_change":"2024-04-23T23:51:50.012988+0000","last_active":"2024-04-23T23:52:09.382726+0000","last_peered":"2024-04-23T23:52:09.382726+0000","last_clean":"2024-04-23T23:52:09.382726+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:09.382726+0000","last_undegraded":"2024-04-23T23:52:09.382726+0000","last_fullsized":"2024-04-23T23:52:09.382726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:48.845107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"129'366","reported_seq":2162,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844525+0000","last_change":"2024-04-23T23:51:50.011281+0000","last_active":"2024-04-23T23:52:07.844525+0000","last_peered":"2024-04-23T23:52:07.844525+0000","last_clean":"2024-04-23T23:52:07.844525+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:07.844525+0000","last_undegraded":"2024-04-23T23:52:07.844525+0000","last_fullsized":"2024-04-23T23:52:07.844525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:11.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"125'277","reported_seq":4560,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108088+0000","last_change":"2024-04-23T23:51:57.042203+0000","last_active":"2024-04-23T23:52:09.108088+0000","last_peered":"2024-04-23T23:52:09.108088+0000","last_clean":"2024-04-23T23:52:09.108088+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:09.108088+0000","last_undegraded":"2024-04-23T23:52:09.108088+0000","last_fullsized":"2024-04-23T23:52:09.108088+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:19.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105223+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:52:09.105223+0000","last_peered":"2024-04-23T23:52:09.105223+0000","last_clean":"2024-04-23T23:52:09.105223+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:09.105223+0000","last_undegraded":"2024-04-23T23:52:09.105223+0000","last_fullsized":"2024-04-23T23:52:09.105223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"131'864","reported_seq":3092,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107941+0000","last_change":"2024-04-23T23:51:57.042767+0000","last_active":"2024-04-23T23:52:09.107941+0000","last_peered":"2024-04-23T23:52:09.107941+0000","last_clean":"2024-04-23T23:52:09.107941+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:09.107941+0000","last_undegraded":"2024-04-23T23:52:09.107941+0000","last_fullsized":"2024-04-23T23:52:09.107941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:28:58.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107742+0000","last_change":"2024-04-23T23:51:53.964555+0000","last_active":"2024-04-23T23:52:09.107742+0000","last_peered":"2024-04-23T23:52:09.107742+0000","last_clean":"2024-04-23T23:52:09.107742+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:09.107742+0000","last_undegraded":"2024-04-23T23:52:09.107742+0000","last_fullsized":"2024-04-23T23:52:09.107742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:05:37.770013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'624","reported_seq":4639,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096820+0000","last_change":"2024-04-23T23:51:50.009306+0000","last_active":"2024-04-23T23:52:10.096820+0000","last_peered":"2024-04-23T23:52:10.096820+0000","last_clean":"2024-04-23T23:52:10.096820+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:10.096820+0000","last_undegraded":"2024-04-23T23:52:10.096820+0000","last_fullsized":"2024-04-23T23:52:10.096820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:34.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":272,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844652+0000","last_change":"2024-04-23T23:51:47.967029+0000","last_active":"2024-04-23T23:52:07.844652+0000","last_peered":"2024-04-23T23:52:07.844652+0000","last_clean":"2024-04-23T23:52:07.844652+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:07.844652+0000","last_undegraded":"2024-04-23T23:52:07.844652+0000","last_fullsized":"2024-04-23T23:52:07.844652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:28.475010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'460","reported_seq":2674,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107932+0000","last_change":"2024-04-23T23:51:57.043327+0000","last_active":"2024-04-23T23:52:09.107932+0000","last_peered":"2024-04-23T23:52:09.107932+0000","last_clean":"2024-04-23T23:52:09.107932+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:09.107932+0000","last_undegraded":"2024-04-23T23:52:09.107932+0000","last_fullsized":"2024-04-23T23:52:09.107932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:35.662689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108102+0000","last_change":"2024-04-23T23:51:53.964521+0000","last_active":"2024-04-23T23:52:09.108102+0000","last_peered":"2024-04-23T23:52:09.108102+0000","last_clean":"2024-04-23T23:52:09.108102+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:09.108102+0000","last_undegraded":"2024-04-23T23:52:09.108102+0000","last_fullsized":"2024-04-23T23:52:09.108102+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:14:20.619635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'541","reported_seq":9060,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382900+0000","last_change":"2024-04-23T23:51:50.014733+0000","last_active":"2024-04-23T23:52:09.382900+0000","last_peered":"2024-04-23T23:52:09.382900+0000","last_clean":"2024-04-23T23:52:09.382900+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:09.382900+0000","last_undegraded":"2024-04-23T23:52:09.382900+0000","last_fullsized":"2024-04-23T23:52:09.382900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:07.885750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"131'742","reported_seq":5032,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092516+0000","last_change":"2024-04-23T23:51:51.016283+0000","last_active":"2024-04-23T23:52:07.092516+0000","last_peered":"2024-04-23T23:52:07.092516+0000","last_clean":"2024-04-23T23:52:07.092516+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:07.092516+0000","last_undegraded":"2024-04-23T23:52:07.092516+0000","last_fullsized":"2024-04-23T23:52:07.092516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:19.673527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382865+0000","last_change":"2024-04-23T23:51:53.965136+0000","last_active":"2024-04-23T23:52:09.382865+0000","last_peered":"2024-04-23T23:52:09.382865+0000","last_clean":"2024-04-23T23:52:09.382865+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:09.382865+0000","last_undegraded":"2024-04-23T23:52:09.382865+0000","last_fullsized":"2024-04-23T23:52:09.382865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:40.481944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"126'317","reported_seq":2745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107507+0000","last_change":"2024-04-23T23:51:57.045408+0000","last_active":"2024-04-23T23:52:09.107507+0000","last_peered":"2024-04-23T23:52:09.107507+0000","last_clean":"2024-04-23T23:52:09.107507+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:09.107507+0000","last_undegraded":"2024-04-23T23:52:09.107507+0000","last_fullsized":"2024-04-23T23:52:09.107507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:34.567554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":271,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.547959+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:52:09.547959+0000","last_peered":"2024-04-23T23:52:09.547959+0000","last_clean":"2024-04-23T23:52:09.547959+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:09.547959+0000","last_undegraded":"2024-04-23T23:52:09.547959+0000","last_fullsized":"2024-04-23T23:52:09.547959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'353","reported_seq":3179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107819+0000","last_change":"2024-04-23T23:51:51.012712+0000","last_active":"2024-04-23T23:52:09.107819+0000","last_peered":"2024-04-23T23:52:09.107819+0000","last_clean":"2024-04-23T23:52:09.107819+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:09.107819+0000","last_undegraded":"2024-04-23T23:52:09.107819+0000","last_fullsized":"2024-04-23T23:52:09.107819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:52.621838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":282,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844466+0000","last_change":"2024-04-23T23:51:53.964436+0000","last_active":"2024-04-23T23:52:07.844466+0000","last_peered":"2024-04-23T23:52:07.844466+0000","last_clean":"2024-04-23T23:52:07.844466+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:07.844466+0000","last_undegraded":"2024-04-23T23:52:07.844466+0000","last_fullsized":"2024-04-23T23:52:07.844466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:59.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"129'630","reported_seq":2024,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096571+0000","last_change":"2024-04-23T23:51:50.007306+0000","last_active":"2024-04-23T23:52:10.096571+0000","last_peered":"2024-04-23T23:52:10.096571+0000","last_clean":"2024-04-23T23:52:10.096571+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:10.096571+0000","last_undegraded":"2024-04-23T23:52:10.096571+0000","last_fullsized":"2024-04-23T23:52:10.096571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:42.222266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382664+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:52:09.382664+0000","last_peered":"2024-04-23T23:52:09.382664+0000","last_clean":"2024-04-23T23:52:09.382664+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:09.382664+0000","last_undegraded":"2024-04-23T23:52:09.382664+0000","last_fullsized":"2024-04-23T23:52:09.382664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":444,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844531+0000","last_change":"2024-04-23T23:51:51.009383+0000","last_active":"2024-04-23T23:52:07.844531+0000","last_peered":"2024-04-23T23:52:07.844531+0000","last_clean":"2024-04-23T23:52:07.844531+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:07.844531+0000","last_undegraded":"2024-04-23T23:52:07.844531+0000","last_fullsized":"2024-04-23T23:52:07.844531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:07.935657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"125'321","reported_seq":1922,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096768+0000","last_change":"2024-04-23T23:51:57.041481+0000","last_active":"2024-04-23T23:52:10.096768+0000","last_peered":"2024-04-23T23:52:10.096768+0000","last_clean":"2024-04-23T23:52:10.096768+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:10.096768+0000","last_undegraded":"2024-04-23T23:52:10.096768+0000","last_fullsized":"2024-04-23T23:52:10.096768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:07.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'516","reported_seq":2892,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092470+0000","last_change":"2024-04-23T23:51:50.014340+0000","last_active":"2024-04-23T23:52:07.092470+0000","last_peered":"2024-04-23T23:52:07.092470+0000","last_clean":"2024-04-23T23:52:07.092470+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:07.092470+0000","last_undegraded":"2024-04-23T23:52:07.092470+0000","last_fullsized":"2024-04-23T23:52:07.092470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:49:40.738996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"129'442","reported_seq":3505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:50.008342+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:00.995303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'443","reported_seq":2952,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097048+0000","last_change":"2024-04-23T23:51:51.016535+0000","last_active":"2024-04-23T23:52:10.097048+0000","last_peered":"2024-04-23T23:52:10.097048+0000","last_clean":"2024-04-23T23:52:10.097048+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:10.097048+0000","last_undegraded":"2024-04-23T23:52:10.097048+0000","last_fullsized":"2024-04-23T23:52:10.097048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:47:29.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'443","reported_seq":4073,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382915+0000","last_change":"2024-04-23T23:51:50.014803+0000","last_active":"2024-04-23T23:52:09.382915+0000","last_peered":"2024-04-23T23:52:09.382915+0000","last_clean":"2024-04-23T23:52:09.382915+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:09.382915+0000","last_undegraded":"2024-04-23T23:52:09.382915+0000","last_fullsized":"2024-04-23T23:52:09.382915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:47.080894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'363","reported_seq":3494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844572+0000","last_change":"2024-04-23T23:51:50.012198+0000","last_active":"2024-04-23T23:52:07.844572+0000","last_peered":"2024-04-23T23:52:07.844572+0000","last_clean":"2024-04-23T23:52:07.844572+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:07.844572+0000","last_undegraded":"2024-04-23T23:52:07.844572+0000","last_fullsized":"2024-04-23T23:52:07.844572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.339718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"123'586","reported_seq":3674,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844493+0000","last_change":"2024-04-23T23:51:50.010749+0000","last_active":"2024-04-23T23:52:07.844493+0000","last_peered":"2024-04-23T23:52:07.844493+0000","last_clean":"2024-04-23T23:52:07.844493+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:07.844493+0000","last_undegraded":"2024-04-23T23:52:07.844493+0000","last_fullsized":"2024-04-23T23:52:07.844493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:23.648951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"131'858","reported_seq":4029,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844441+0000","last_change":"2024-04-23T23:51:50.010902+0000","last_active":"2024-04-23T23:52:07.844441+0000","last_peered":"2024-04-23T23:52:07.844441+0000","last_clean":"2024-04-23T23:52:07.844441+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:07.844441+0000","last_undegraded":"2024-04-23T23:52:07.844441+0000","last_fullsized":"2024-04-23T23:52:07.844441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:29.439340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"129'321","reported_seq":5844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:51.012948+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:42.912702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"118'411","reported_seq":4195,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844358+0000","last_change":"2024-04-23T23:51:50.007735+0000","last_active":"2024-04-23T23:52:07.844358+0000","last_peered":"2024-04-23T23:52:07.844358+0000","last_clean":"2024-04-23T23:52:07.844358+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:07.844358+0000","last_undegraded":"2024-04-23T23:52:07.844358+0000","last_fullsized":"2024-04-23T23:52:07.844358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:10:47.430347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"126'451","reported_seq":2347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382665+0000","last_change":"2024-04-23T23:51:50.011706+0000","last_active":"2024-04-23T23:52:09.382665+0000","last_peered":"2024-04-23T23:52:09.382665+0000","last_clean":"2024-04-23T23:52:09.382665+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:09.382665+0000","last_undegraded":"2024-04-23T23:52:09.382665+0000","last_fullsized":"2024-04-23T23:52:09.382665+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:50.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"126'486","reported_seq":3064,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844367+0000","last_change":"2024-04-23T23:51:50.011755+0000","last_active":"2024-04-23T23:52:07.844367+0000","last_peered":"2024-04-23T23:52:07.844367+0000","last_clean":"2024-04-23T23:52:07.844367+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:07.844367+0000","last_undegraded":"2024-04-23T23:52:07.844367+0000","last_fullsized":"2024-04-23T23:52:07.844367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:45:45.614364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16569,"ondisk_log_size":16569,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168598,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46924,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324916,"statfs":{"total":96636764160,"available":96588713984,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2338816,"data_stored":1427202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50936,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320904,"statfs":{"total":96636764160,"available":96584605696,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":836853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:13.985 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T23:52:13.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-23T23:52:14.288 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T23:52:14.299 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T23:52:14.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:14.581 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:14.592 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T23:52:13.383614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16686,"ondisk_log_size":16686,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389416,"kb_used_data":14896,"kb_used_omap":21,"kb_used_meta":374378,"kb_avail":754585304,"statfs":{"total":773094113280,"available":772695351296,"internally_reserved":0,"allocated":15253504,"data_stored":7922414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":383363445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.292260"},"pg_stats":[{"pgid":"3.1f","version":"131'661","reported_seq":8954,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107934+0000","last_change":"2024-04-23T23:51:51.018566+0000","last_active":"2024-04-23T23:52:09.107934+0000","last_peered":"2024-04-23T23:52:09.107934+0000","last_clean":"2024-04-23T23:52:09.107934+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:09.107934+0000","last_undegraded":"2024-04-23T23:52:09.107934+0000","last_fullsized":"2024-04-23T23:52:09.107934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:45.446677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'834","reported_seq":4329,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096720+0000","last_change":"2024-04-23T23:51:50.007789+0000","last_active":"2024-04-23T23:52:10.096720+0000","last_peered":"2024-04-23T23:52:10.096720+0000","last_clean":"2024-04-23T23:52:10.096720+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:10.096720+0000","last_undegraded":"2024-04-23T23:52:10.096720+0000","last_fullsized":"2024-04-23T23:52:10.096720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:44:24.111886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"119'379","reported_seq":3308,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382998+0000","last_change":"2024-04-23T23:51:50.014697+0000","last_active":"2024-04-23T23:52:09.382998+0000","last_peered":"2024-04-23T23:52:09.382998+0000","last_clean":"2024-04-23T23:52:09.382998+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:09.382998+0000","last_undegraded":"2024-04-23T23:52:09.382998+0000","last_fullsized":"2024-04-23T23:52:09.382998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:40.316577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"131'741","reported_seq":6498,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.383104+0000","last_change":"2024-04-23T23:51:50.013253+0000","last_active":"2024-04-23T23:52:09.383104+0000","last_peered":"2024-04-23T23:52:09.383104+0000","last_clean":"2024-04-23T23:52:09.383104+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:09.383104+0000","last_undegraded":"2024-04-23T23:52:09.383104+0000","last_fullsized":"2024-04-23T23:52:09.383104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:47.200572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"129'304","reported_seq":4086,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107808+0000","last_change":"2024-04-23T23:51:51.017335+0000","last_active":"2024-04-23T23:52:09.107808+0000","last_peered":"2024-04-23T23:52:09.107808+0000","last_clean":"2024-04-23T23:52:09.107808+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:09.107808+0000","last_undegraded":"2024-04-23T23:52:09.107808+0000","last_fullsized":"2024-04-23T23:52:09.107808+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:52:41.586532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"123'333","reported_seq":4979,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092378+0000","last_change":"2024-04-23T23:51:57.043361+0000","last_active":"2024-04-23T23:52:07.092378+0000","last_peered":"2024-04-23T23:52:07.092378+0000","last_clean":"2024-04-23T23:52:07.092378+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:07.092378+0000","last_undegraded":"2024-04-23T23:52:07.092378+0000","last_fullsized":"2024-04-23T23:52:07.092378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:06.039919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"119'788","reported_seq":6488,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107545+0000","last_change":"2024-04-23T23:51:57.044986+0000","last_active":"2024-04-23T23:52:09.107545+0000","last_peered":"2024-04-23T23:52:09.107545+0000","last_clean":"2024-04-23T23:52:09.107545+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:09.107545+0000","last_undegraded":"2024-04-23T23:52:09.107545+0000","last_fullsized":"2024-04-23T23:52:09.107545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:22.571801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"126'626","reported_seq":3564,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097250+0000","last_change":"2024-04-23T23:51:51.014869+0000","last_active":"2024-04-23T23:52:10.097250+0000","last_peered":"2024-04-23T23:52:10.097250+0000","last_clean":"2024-04-23T23:52:10.097250+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:10.097250+0000","last_undegraded":"2024-04-23T23:52:10.097250+0000","last_fullsized":"2024-04-23T23:52:10.097250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:25.995036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"131'620","reported_seq":6367,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107990+0000","last_change":"2024-04-23T23:51:51.021741+0000","last_active":"2024-04-23T23:52:09.107990+0000","last_peered":"2024-04-23T23:52:09.107990+0000","last_clean":"2024-04-23T23:52:09.107990+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:09.107990+0000","last_undegraded":"2024-04-23T23:52:09.107990+0000","last_fullsized":"2024-04-23T23:52:09.107990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:28:57.626808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"131'468","reported_seq":6523,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382726+0000","last_change":"2024-04-23T23:51:50.012988+0000","last_active":"2024-04-23T23:52:09.382726+0000","last_peered":"2024-04-23T23:52:09.382726+0000","last_clean":"2024-04-23T23:52:09.382726+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:09.382726+0000","last_undegraded":"2024-04-23T23:52:09.382726+0000","last_fullsized":"2024-04-23T23:52:09.382726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:48.845107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"129'366","reported_seq":2162,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844525+0000","last_change":"2024-04-23T23:51:50.011281+0000","last_active":"2024-04-23T23:52:07.844525+0000","last_peered":"2024-04-23T23:52:07.844525+0000","last_clean":"2024-04-23T23:52:07.844525+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:07.844525+0000","last_undegraded":"2024-04-23T23:52:07.844525+0000","last_fullsized":"2024-04-23T23:52:07.844525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:11.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"125'277","reported_seq":4560,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108088+0000","last_change":"2024-04-23T23:51:57.042203+0000","last_active":"2024-04-23T23:52:09.108088+0000","last_peered":"2024-04-23T23:52:09.108088+0000","last_clean":"2024-04-23T23:52:09.108088+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:09.108088+0000","last_undegraded":"2024-04-23T23:52:09.108088+0000","last_fullsized":"2024-04-23T23:52:09.108088+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:19.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105223+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:52:09.105223+0000","last_peered":"2024-04-23T23:52:09.105223+0000","last_clean":"2024-04-23T23:52:09.105223+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:09.105223+0000","last_undegraded":"2024-04-23T23:52:09.105223+0000","last_fullsized":"2024-04-23T23:52:09.105223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"131'864","reported_seq":3092,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107941+0000","last_change":"2024-04-23T23:51:57.042767+0000","last_active":"2024-04-23T23:52:09.107941+0000","last_peered":"2024-04-23T23:52:09.107941+0000","last_clean":"2024-04-23T23:52:09.107941+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:09.107941+0000","last_undegraded":"2024-04-23T23:52:09.107941+0000","last_fullsized":"2024-04-23T23:52:09.107941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:28:58.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107742+0000","last_change":"2024-04-23T23:51:53.964555+0000","last_active":"2024-04-23T23:52:09.107742+0000","last_peered":"2024-04-23T23:52:09.107742+0000","last_clean":"2024-04-23T23:52:09.107742+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:09.107742+0000","last_undegraded":"2024-04-23T23:52:09.107742+0000","last_fullsized":"2024-04-23T23:52:09.107742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:05:37.770013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'624","reported_seq":4639,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096820+0000","last_change":"2024-04-23T23:51:50.009306+0000","last_active":"2024-04-23T23:52:10.096820+0000","last_peered":"2024-04-23T23:52:10.096820+0000","last_clean":"2024-04-23T23:52:10.096820+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:10.096820+0000","last_undegraded":"2024-04-23T23:52:10.096820+0000","last_fullsized":"2024-04-23T23:52:10.096820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:34.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":272,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844652+0000","last_change":"2024-04-23T23:51:47.967029+0000","last_active":"2024-04-23T23:52:07.844652+0000","last_peered":"2024-04-23T23:52:07.844652+0000","last_clean":"2024-04-23T23:52:07.844652+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:07.844652+0000","last_undegraded":"2024-04-23T23:52:07.844652+0000","last_fullsized":"2024-04-23T23:52:07.844652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:28.475010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'460","reported_seq":2674,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107932+0000","last_change":"2024-04-23T23:51:57.043327+0000","last_active":"2024-04-23T23:52:09.107932+0000","last_peered":"2024-04-23T23:52:09.107932+0000","last_clean":"2024-04-23T23:52:09.107932+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:09.107932+0000","last_undegraded":"2024-04-23T23:52:09.107932+0000","last_fullsized":"2024-04-23T23:52:09.107932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:35.662689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108102+0000","last_change":"2024-04-23T23:51:53.964521+0000","last_active":"2024-04-23T23:52:09.108102+0000","last_peered":"2024-04-23T23:52:09.108102+0000","last_clean":"2024-04-23T23:52:09.108102+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:09.108102+0000","last_undegraded":"2024-04-23T23:52:09.108102+0000","last_fullsized":"2024-04-23T23:52:09.108102+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:14:20.619635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'541","reported_seq":9060,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382900+0000","last_change":"2024-04-23T23:51:50.014733+0000","last_active":"2024-04-23T23:52:09.382900+0000","last_peered":"2024-04-23T23:52:09.382900+0000","last_clean":"2024-04-23T23:52:09.382900+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:09.382900+0000","last_undegraded":"2024-04-23T23:52:09.382900+0000","last_fullsized":"2024-04-23T23:52:09.382900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:07.885750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"131'742","reported_seq":5032,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092516+0000","last_change":"2024-04-23T23:51:51.016283+0000","last_active":"2024-04-23T23:52:07.092516+0000","last_peered":"2024-04-23T23:52:07.092516+0000","last_clean":"2024-04-23T23:52:07.092516+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:07.092516+0000","last_undegraded":"2024-04-23T23:52:07.092516+0000","last_fullsized":"2024-04-23T23:52:07.092516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:19.673527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382865+0000","last_change":"2024-04-23T23:51:53.965136+0000","last_active":"2024-04-23T23:52:09.382865+0000","last_peered":"2024-04-23T23:52:09.382865+0000","last_clean":"2024-04-23T23:52:09.382865+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:09.382865+0000","last_undegraded":"2024-04-23T23:52:09.382865+0000","last_fullsized":"2024-04-23T23:52:09.382865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:40.481944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"126'317","reported_seq":2745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107507+0000","last_change":"2024-04-23T23:51:57.045408+0000","last_active":"2024-04-23T23:52:09.107507+0000","last_peered":"2024-04-23T23:52:09.107507+0000","last_clean":"2024-04-23T23:52:09.107507+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:09.107507+0000","last_undegraded":"2024-04-23T23:52:09.107507+0000","last_fullsized":"2024-04-23T23:52:09.107507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:34.567554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":271,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.547959+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:52:09.547959+0000","last_peered":"2024-04-23T23:52:09.547959+0000","last_clean":"2024-04-23T23:52:09.547959+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:09.547959+0000","last_undegraded":"2024-04-23T23:52:09.547959+0000","last_fullsized":"2024-04-23T23:52:09.547959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'353","reported_seq":3179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107819+0000","last_change":"2024-04-23T23:51:51.012712+0000","last_active":"2024-04-23T23:52:09.107819+0000","last_peered":"2024-04-23T23:52:09.107819+0000","last_clean":"2024-04-23T23:52:09.107819+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:09.107819+0000","last_undegraded":"2024-04-23T23:52:09.107819+0000","last_fullsized":"2024-04-23T23:52:09.107819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:52.621838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":282,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844466+0000","last_change":"2024-04-23T23:51:53.964436+0000","last_active":"2024-04-23T23:52:07.844466+0000","last_peered":"2024-04-23T23:52:07.844466+0000","last_clean":"2024-04-23T23:52:07.844466+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:07.844466+0000","last_undegraded":"2024-04-23T23:52:07.844466+0000","last_fullsized":"2024-04-23T23:52:07.844466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:59.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"129'630","reported_seq":2024,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096571+0000","last_change":"2024-04-23T23:51:50.007306+0000","last_active":"2024-04-23T23:52:10.096571+0000","last_peered":"2024-04-23T23:52:10.096571+0000","last_clean":"2024-04-23T23:52:10.096571+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:10.096571+0000","last_undegraded":"2024-04-23T23:52:10.096571+0000","last_fullsized":"2024-04-23T23:52:10.096571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:42.222266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382664+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:52:09.382664+0000","last_peered":"2024-04-23T23:52:09.382664+0000","last_clean":"2024-04-23T23:52:09.382664+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:09.382664+0000","last_undegraded":"2024-04-23T23:52:09.382664+0000","last_fullsized":"2024-04-23T23:52:09.382664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":444,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844531+0000","last_change":"2024-04-23T23:51:51.009383+0000","last_active":"2024-04-23T23:52:07.844531+0000","last_peered":"2024-04-23T23:52:07.844531+0000","last_clean":"2024-04-23T23:52:07.844531+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:07.844531+0000","last_undegraded":"2024-04-23T23:52:07.844531+0000","last_fullsized":"2024-04-23T23:52:07.844531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:07.935657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"125'321","reported_seq":1922,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096768+0000","last_change":"2024-04-23T23:51:57.041481+0000","last_active":"2024-04-23T23:52:10.096768+0000","last_peered":"2024-04-23T23:52:10.096768+0000","last_clean":"2024-04-23T23:52:10.096768+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:10.096768+0000","last_undegraded":"2024-04-23T23:52:10.096768+0000","last_fullsized":"2024-04-23T23:52:10.096768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:07.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'516","reported_seq":2892,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092470+0000","last_change":"2024-04-23T23:51:50.014340+0000","last_active":"2024-04-23T23:52:07.092470+0000","last_peered":"2024-04-23T23:52:07.092470+0000","last_clean":"2024-04-23T23:52:07.092470+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:07.092470+0000","last_undegraded":"2024-04-23T23:52:07.092470+0000","last_fullsized":"2024-04-23T23:52:07.092470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:49:40.738996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"129'442","reported_seq":3505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:50.008342+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:00.995303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'443","reported_seq":2952,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097048+0000","last_change":"2024-04-23T23:51:51.016535+0000","last_active":"2024-04-23T23:52:10.097048+0000","last_peered":"2024-04-23T23:52:10.097048+0000","last_clean":"2024-04-23T23:52:10.097048+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:10.097048+0000","last_undegraded":"2024-04-23T23:52:10.097048+0000","last_fullsized":"2024-04-23T23:52:10.097048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:47:29.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'443","reported_seq":4073,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382915+0000","last_change":"2024-04-23T23:51:50.014803+0000","last_active":"2024-04-23T23:52:09.382915+0000","last_peered":"2024-04-23T23:52:09.382915+0000","last_clean":"2024-04-23T23:52:09.382915+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:09.382915+0000","last_undegraded":"2024-04-23T23:52:09.382915+0000","last_fullsized":"2024-04-23T23:52:09.382915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:47.080894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'363","reported_seq":3494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844572+0000","last_change":"2024-04-23T23:51:50.012198+0000","last_active":"2024-04-23T23:52:07.844572+0000","last_peered":"2024-04-23T23:52:07.844572+0000","last_clean":"2024-04-23T23:52:07.844572+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:07.844572+0000","last_undegraded":"2024-04-23T23:52:07.844572+0000","last_fullsized":"2024-04-23T23:52:07.844572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.339718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"123'586","reported_seq":3674,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844493+0000","last_change":"2024-04-23T23:51:50.010749+0000","last_active":"2024-04-23T23:52:07.844493+0000","last_peered":"2024-04-23T23:52:07.844493+0000","last_clean":"2024-04-23T23:52:07.844493+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:07.844493+0000","last_undegraded":"2024-04-23T23:52:07.844493+0000","last_fullsized":"2024-04-23T23:52:07.844493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:23.648951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"131'858","reported_seq":4029,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844441+0000","last_change":"2024-04-23T23:51:50.010902+0000","last_active":"2024-04-23T23:52:07.844441+0000","last_peered":"2024-04-23T23:52:07.844441+0000","last_clean":"2024-04-23T23:52:07.844441+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:07.844441+0000","last_undegraded":"2024-04-23T23:52:07.844441+0000","last_fullsized":"2024-04-23T23:52:07.844441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:29.439340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"129'321","reported_seq":5844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:51.012948+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:42.912702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"118'411","reported_seq":4195,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844358+0000","last_change":"2024-04-23T23:51:50.007735+0000","last_active":"2024-04-23T23:52:07.844358+0000","last_peered":"2024-04-23T23:52:07.844358+0000","last_clean":"2024-04-23T23:52:07.844358+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:07.844358+0000","last_undegraded":"2024-04-23T23:52:07.844358+0000","last_fullsized":"2024-04-23T23:52:07.844358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:10:47.430347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"126'451","reported_seq":2347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382665+0000","last_change":"2024-04-23T23:51:50.011706+0000","last_active":"2024-04-23T23:52:09.382665+0000","last_peered":"2024-04-23T23:52:09.382665+0000","last_clean":"2024-04-23T23:52:09.382665+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:09.382665+0000","last_undegraded":"2024-04-23T23:52:09.382665+0000","last_fullsized":"2024-04-23T23:52:09.382665+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:50.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"126'486","reported_seq":3064,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844367+0000","last_change":"2024-04-23T23:51:50.011755+0000","last_active":"2024-04-23T23:52:07.844367+0000","last_peered":"2024-04-23T23:52:07.844367+0000","last_clean":"2024-04-23T23:52:07.844367+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:07.844367+0000","last_undegraded":"2024-04-23T23:52:07.844367+0000","last_fullsized":"2024-04-23T23:52:07.844367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:45:45.614364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16569,"ondisk_log_size":16569,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168598,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46924,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324916,"statfs":{"total":96636764160,"available":96588713984,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2338816,"data_stored":1427202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50936,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320904,"statfs":{"total":96636764160,"available":96584605696,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":836853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:14.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:14.874 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:14.875 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:14.886 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T23:52:13.383614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16686,"ondisk_log_size":16686,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389416,"kb_used_data":14896,"kb_used_omap":21,"kb_used_meta":374378,"kb_avail":754585304,"statfs":{"total":773094113280,"available":772695351296,"internally_reserved":0,"allocated":15253504,"data_stored":7922414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":383363445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.292260"},"pg_stats":[{"pgid":"3.1f","version":"131'661","reported_seq":8954,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107934+0000","last_change":"2024-04-23T23:51:51.018566+0000","last_active":"2024-04-23T23:52:09.107934+0000","last_peered":"2024-04-23T23:52:09.107934+0000","last_clean":"2024-04-23T23:52:09.107934+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:09.107934+0000","last_undegraded":"2024-04-23T23:52:09.107934+0000","last_fullsized":"2024-04-23T23:52:09.107934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:45.446677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'834","reported_seq":4329,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096720+0000","last_change":"2024-04-23T23:51:50.007789+0000","last_active":"2024-04-23T23:52:10.096720+0000","last_peered":"2024-04-23T23:52:10.096720+0000","last_clean":"2024-04-23T23:52:10.096720+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:10.096720+0000","last_undegraded":"2024-04-23T23:52:10.096720+0000","last_fullsized":"2024-04-23T23:52:10.096720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:44:24.111886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"119'379","reported_seq":3308,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382998+0000","last_change":"2024-04-23T23:51:50.014697+0000","last_active":"2024-04-23T23:52:09.382998+0000","last_peered":"2024-04-23T23:52:09.382998+0000","last_clean":"2024-04-23T23:52:09.382998+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:09.382998+0000","last_undegraded":"2024-04-23T23:52:09.382998+0000","last_fullsized":"2024-04-23T23:52:09.382998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:40.316577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"131'741","reported_seq":6498,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.383104+0000","last_change":"2024-04-23T23:51:50.013253+0000","last_active":"2024-04-23T23:52:09.383104+0000","last_peered":"2024-04-23T23:52:09.383104+0000","last_clean":"2024-04-23T23:52:09.383104+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:09.383104+0000","last_undegraded":"2024-04-23T23:52:09.383104+0000","last_fullsized":"2024-04-23T23:52:09.383104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:47.200572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"129'304","reported_seq":4086,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107808+0000","last_change":"2024-04-23T23:51:51.017335+0000","last_active":"2024-04-23T23:52:09.107808+0000","last_peered":"2024-04-23T23:52:09.107808+0000","last_clean":"2024-04-23T23:52:09.107808+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:09.107808+0000","last_undegraded":"2024-04-23T23:52:09.107808+0000","last_fullsized":"2024-04-23T23:52:09.107808+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:52:41.586532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"123'333","reported_seq":4979,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092378+0000","last_change":"2024-04-23T23:51:57.043361+0000","last_active":"2024-04-23T23:52:07.092378+0000","last_peered":"2024-04-23T23:52:07.092378+0000","last_clean":"2024-04-23T23:52:07.092378+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:07.092378+0000","last_undegraded":"2024-04-23T23:52:07.092378+0000","last_fullsized":"2024-04-23T23:52:07.092378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:06.039919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"119'788","reported_seq":6488,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107545+0000","last_change":"2024-04-23T23:51:57.044986+0000","last_active":"2024-04-23T23:52:09.107545+0000","last_peered":"2024-04-23T23:52:09.107545+0000","last_clean":"2024-04-23T23:52:09.107545+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:09.107545+0000","last_undegraded":"2024-04-23T23:52:09.107545+0000","last_fullsized":"2024-04-23T23:52:09.107545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:22.571801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"126'626","reported_seq":3564,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097250+0000","last_change":"2024-04-23T23:51:51.014869+0000","last_active":"2024-04-23T23:52:10.097250+0000","last_peered":"2024-04-23T23:52:10.097250+0000","last_clean":"2024-04-23T23:52:10.097250+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:10.097250+0000","last_undegraded":"2024-04-23T23:52:10.097250+0000","last_fullsized":"2024-04-23T23:52:10.097250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:25.995036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"131'620","reported_seq":6367,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107990+0000","last_change":"2024-04-23T23:51:51.021741+0000","last_active":"2024-04-23T23:52:09.107990+0000","last_peered":"2024-04-23T23:52:09.107990+0000","last_clean":"2024-04-23T23:52:09.107990+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:09.107990+0000","last_undegraded":"2024-04-23T23:52:09.107990+0000","last_fullsized":"2024-04-23T23:52:09.107990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:28:57.626808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"131'468","reported_seq":6523,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382726+0000","last_change":"2024-04-23T23:51:50.012988+0000","last_active":"2024-04-23T23:52:09.382726+0000","last_peered":"2024-04-23T23:52:09.382726+0000","last_clean":"2024-04-23T23:52:09.382726+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:09.382726+0000","last_undegraded":"2024-04-23T23:52:09.382726+0000","last_fullsized":"2024-04-23T23:52:09.382726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:48.845107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"129'366","reported_seq":2162,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844525+0000","last_change":"2024-04-23T23:51:50.011281+0000","last_active":"2024-04-23T23:52:07.844525+0000","last_peered":"2024-04-23T23:52:07.844525+0000","last_clean":"2024-04-23T23:52:07.844525+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:07.844525+0000","last_undegraded":"2024-04-23T23:52:07.844525+0000","last_fullsized":"2024-04-23T23:52:07.844525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:11.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"125'277","reported_seq":4560,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108088+0000","last_change":"2024-04-23T23:51:57.042203+0000","last_active":"2024-04-23T23:52:09.108088+0000","last_peered":"2024-04-23T23:52:09.108088+0000","last_clean":"2024-04-23T23:52:09.108088+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:09.108088+0000","last_undegraded":"2024-04-23T23:52:09.108088+0000","last_fullsized":"2024-04-23T23:52:09.108088+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:19.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105223+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:52:09.105223+0000","last_peered":"2024-04-23T23:52:09.105223+0000","last_clean":"2024-04-23T23:52:09.105223+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:09.105223+0000","last_undegraded":"2024-04-23T23:52:09.105223+0000","last_fullsized":"2024-04-23T23:52:09.105223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"131'864","reported_seq":3092,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107941+0000","last_change":"2024-04-23T23:51:57.042767+0000","last_active":"2024-04-23T23:52:09.107941+0000","last_peered":"2024-04-23T23:52:09.107941+0000","last_clean":"2024-04-23T23:52:09.107941+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:09.107941+0000","last_undegraded":"2024-04-23T23:52:09.107941+0000","last_fullsized":"2024-04-23T23:52:09.107941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:28:58.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107742+0000","last_change":"2024-04-23T23:51:53.964555+0000","last_active":"2024-04-23T23:52:09.107742+0000","last_peered":"2024-04-23T23:52:09.107742+0000","last_clean":"2024-04-23T23:52:09.107742+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:09.107742+0000","last_undegraded":"2024-04-23T23:52:09.107742+0000","last_fullsized":"2024-04-23T23:52:09.107742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:05:37.770013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'624","reported_seq":4639,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096820+0000","last_change":"2024-04-23T23:51:50.009306+0000","last_active":"2024-04-23T23:52:10.096820+0000","last_peered":"2024-04-23T23:52:10.096820+0000","last_clean":"2024-04-23T23:52:10.096820+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:10.096820+0000","last_undegraded":"2024-04-23T23:52:10.096820+0000","last_fullsized":"2024-04-23T23:52:10.096820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:34.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":272,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844652+0000","last_change":"2024-04-23T23:51:47.967029+0000","last_active":"2024-04-23T23:52:07.844652+0000","last_peered":"2024-04-23T23:52:07.844652+0000","last_clean":"2024-04-23T23:52:07.844652+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:07.844652+0000","last_undegraded":"2024-04-23T23:52:07.844652+0000","last_fullsized":"2024-04-23T23:52:07.844652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:28.475010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'460","reported_seq":2674,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107932+0000","last_change":"2024-04-23T23:51:57.043327+0000","last_active":"2024-04-23T23:52:09.107932+0000","last_peered":"2024-04-23T23:52:09.107932+0000","last_clean":"2024-04-23T23:52:09.107932+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:09.107932+0000","last_undegraded":"2024-04-23T23:52:09.107932+0000","last_fullsized":"2024-04-23T23:52:09.107932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:35.662689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108102+0000","last_change":"2024-04-23T23:51:53.964521+0000","last_active":"2024-04-23T23:52:09.108102+0000","last_peered":"2024-04-23T23:52:09.108102+0000","last_clean":"2024-04-23T23:52:09.108102+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:09.108102+0000","last_undegraded":"2024-04-23T23:52:09.108102+0000","last_fullsized":"2024-04-23T23:52:09.108102+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:14:20.619635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'541","reported_seq":9060,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382900+0000","last_change":"2024-04-23T23:51:50.014733+0000","last_active":"2024-04-23T23:52:09.382900+0000","last_peered":"2024-04-23T23:52:09.382900+0000","last_clean":"2024-04-23T23:52:09.382900+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:09.382900+0000","last_undegraded":"2024-04-23T23:52:09.382900+0000","last_fullsized":"2024-04-23T23:52:09.382900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:07.885750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"131'742","reported_seq":5032,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092516+0000","last_change":"2024-04-23T23:51:51.016283+0000","last_active":"2024-04-23T23:52:07.092516+0000","last_peered":"2024-04-23T23:52:07.092516+0000","last_clean":"2024-04-23T23:52:07.092516+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:07.092516+0000","last_undegraded":"2024-04-23T23:52:07.092516+0000","last_fullsized":"2024-04-23T23:52:07.092516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:19.673527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382865+0000","last_change":"2024-04-23T23:51:53.965136+0000","last_active":"2024-04-23T23:52:09.382865+0000","last_peered":"2024-04-23T23:52:09.382865+0000","last_clean":"2024-04-23T23:52:09.382865+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:09.382865+0000","last_undegraded":"2024-04-23T23:52:09.382865+0000","last_fullsized":"2024-04-23T23:52:09.382865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:40.481944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"126'317","reported_seq":2745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107507+0000","last_change":"2024-04-23T23:51:57.045408+0000","last_active":"2024-04-23T23:52:09.107507+0000","last_peered":"2024-04-23T23:52:09.107507+0000","last_clean":"2024-04-23T23:52:09.107507+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:09.107507+0000","last_undegraded":"2024-04-23T23:52:09.107507+0000","last_fullsized":"2024-04-23T23:52:09.107507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:34.567554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":271,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.547959+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:52:09.547959+0000","last_peered":"2024-04-23T23:52:09.547959+0000","last_clean":"2024-04-23T23:52:09.547959+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:09.547959+0000","last_undegraded":"2024-04-23T23:52:09.547959+0000","last_fullsized":"2024-04-23T23:52:09.547959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'353","reported_seq":3179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107819+0000","last_change":"2024-04-23T23:51:51.012712+0000","last_active":"2024-04-23T23:52:09.107819+0000","last_peered":"2024-04-23T23:52:09.107819+0000","last_clean":"2024-04-23T23:52:09.107819+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:09.107819+0000","last_undegraded":"2024-04-23T23:52:09.107819+0000","last_fullsized":"2024-04-23T23:52:09.107819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:52.621838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":282,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844466+0000","last_change":"2024-04-23T23:51:53.964436+0000","last_active":"2024-04-23T23:52:07.844466+0000","last_peered":"2024-04-23T23:52:07.844466+0000","last_clean":"2024-04-23T23:52:07.844466+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:07.844466+0000","last_undegraded":"2024-04-23T23:52:07.844466+0000","last_fullsized":"2024-04-23T23:52:07.844466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:59.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"129'630","reported_seq":2024,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096571+0000","last_change":"2024-04-23T23:51:50.007306+0000","last_active":"2024-04-23T23:52:10.096571+0000","last_peered":"2024-04-23T23:52:10.096571+0000","last_clean":"2024-04-23T23:52:10.096571+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:10.096571+0000","last_undegraded":"2024-04-23T23:52:10.096571+0000","last_fullsized":"2024-04-23T23:52:10.096571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:42.222266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382664+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:52:09.382664+0000","last_peered":"2024-04-23T23:52:09.382664+0000","last_clean":"2024-04-23T23:52:09.382664+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:09.382664+0000","last_undegraded":"2024-04-23T23:52:09.382664+0000","last_fullsized":"2024-04-23T23:52:09.382664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":444,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844531+0000","last_change":"2024-04-23T23:51:51.009383+0000","last_active":"2024-04-23T23:52:07.844531+0000","last_peered":"2024-04-23T23:52:07.844531+0000","last_clean":"2024-04-23T23:52:07.844531+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:07.844531+0000","last_undegraded":"2024-04-23T23:52:07.844531+0000","last_fullsized":"2024-04-23T23:52:07.844531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:07.935657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"125'321","reported_seq":1922,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096768+0000","last_change":"2024-04-23T23:51:57.041481+0000","last_active":"2024-04-23T23:52:10.096768+0000","last_peered":"2024-04-23T23:52:10.096768+0000","last_clean":"2024-04-23T23:52:10.096768+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:10.096768+0000","last_undegraded":"2024-04-23T23:52:10.096768+0000","last_fullsized":"2024-04-23T23:52:10.096768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:07.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'516","reported_seq":2892,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092470+0000","last_change":"2024-04-23T23:51:50.014340+0000","last_active":"2024-04-23T23:52:07.092470+0000","last_peered":"2024-04-23T23:52:07.092470+0000","last_clean":"2024-04-23T23:52:07.092470+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:07.092470+0000","last_undegraded":"2024-04-23T23:52:07.092470+0000","last_fullsized":"2024-04-23T23:52:07.092470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:49:40.738996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"129'442","reported_seq":3505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:50.008342+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:00.995303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'443","reported_seq":2952,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097048+0000","last_change":"2024-04-23T23:51:51.016535+0000","last_active":"2024-04-23T23:52:10.097048+0000","last_peered":"2024-04-23T23:52:10.097048+0000","last_clean":"2024-04-23T23:52:10.097048+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:10.097048+0000","last_undegraded":"2024-04-23T23:52:10.097048+0000","last_fullsized":"2024-04-23T23:52:10.097048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:47:29.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'443","reported_seq":4073,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382915+0000","last_change":"2024-04-23T23:51:50.014803+0000","last_active":"2024-04-23T23:52:09.382915+0000","last_peered":"2024-04-23T23:52:09.382915+0000","last_clean":"2024-04-23T23:52:09.382915+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:09.382915+0000","last_undegraded":"2024-04-23T23:52:09.382915+0000","last_fullsized":"2024-04-23T23:52:09.382915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:47.080894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'363","reported_seq":3494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844572+0000","last_change":"2024-04-23T23:51:50.012198+0000","last_active":"2024-04-23T23:52:07.844572+0000","last_peered":"2024-04-23T23:52:07.844572+0000","last_clean":"2024-04-23T23:52:07.844572+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:07.844572+0000","last_undegraded":"2024-04-23T23:52:07.844572+0000","last_fullsized":"2024-04-23T23:52:07.844572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.339718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"123'586","reported_seq":3674,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844493+0000","last_change":"2024-04-23T23:51:50.010749+0000","last_active":"2024-04-23T23:52:07.844493+0000","last_peered":"2024-04-23T23:52:07.844493+0000","last_clean":"2024-04-23T23:52:07.844493+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:07.844493+0000","last_undegraded":"2024-04-23T23:52:07.844493+0000","last_fullsized":"2024-04-23T23:52:07.844493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:23.648951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"131'858","reported_seq":4029,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844441+0000","last_change":"2024-04-23T23:51:50.010902+0000","last_active":"2024-04-23T23:52:07.844441+0000","last_peered":"2024-04-23T23:52:07.844441+0000","last_clean":"2024-04-23T23:52:07.844441+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:07.844441+0000","last_undegraded":"2024-04-23T23:52:07.844441+0000","last_fullsized":"2024-04-23T23:52:07.844441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:29.439340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"129'321","reported_seq":5844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:51.012948+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:42.912702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"118'411","reported_seq":4195,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844358+0000","last_change":"2024-04-23T23:51:50.007735+0000","last_active":"2024-04-23T23:52:07.844358+0000","last_peered":"2024-04-23T23:52:07.844358+0000","last_clean":"2024-04-23T23:52:07.844358+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:07.844358+0000","last_undegraded":"2024-04-23T23:52:07.844358+0000","last_fullsized":"2024-04-23T23:52:07.844358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:10:47.430347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"126'451","reported_seq":2347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382665+0000","last_change":"2024-04-23T23:51:50.011706+0000","last_active":"2024-04-23T23:52:09.382665+0000","last_peered":"2024-04-23T23:52:09.382665+0000","last_clean":"2024-04-23T23:52:09.382665+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:09.382665+0000","last_undegraded":"2024-04-23T23:52:09.382665+0000","last_fullsized":"2024-04-23T23:52:09.382665+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:50.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"126'486","reported_seq":3064,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844367+0000","last_change":"2024-04-23T23:51:50.011755+0000","last_active":"2024-04-23T23:52:07.844367+0000","last_peered":"2024-04-23T23:52:07.844367+0000","last_clean":"2024-04-23T23:52:07.844367+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:07.844367+0000","last_undegraded":"2024-04-23T23:52:07.844367+0000","last_fullsized":"2024-04-23T23:52:07.844367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:45:45.614364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16569,"ondisk_log_size":16569,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168598,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46924,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324916,"statfs":{"total":96636764160,"available":96588713984,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2338816,"data_stored":1427202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50936,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320904,"statfs":{"total":96636764160,"available":96584605696,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":836853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:14.888 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T23:52:14.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:15.185 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T23:52:13.383614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16686,"ondisk_log_size":16686,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389416,"kb_used_data":14896,"kb_used_omap":21,"kb_used_meta":374378,"kb_avail":754585304,"statfs":{"total":773094113280,"available":772695351296,"internally_reserved":0,"allocated":15253504,"data_stored":7922414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":383363445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.292260"},"pg_stats":[{"pgid":"3.1f","version":"131'661","reported_seq":8954,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107934+0000","last_change":"2024-04-23T23:51:51.018566+0000","last_active":"2024-04-23T23:52:09.107934+0000","last_peered":"2024-04-23T23:52:09.107934+0000","last_clean":"2024-04-23T23:52:09.107934+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:09.107934+0000","last_undegraded":"2024-04-23T23:52:09.107934+0000","last_fullsized":"2024-04-23T23:52:09.107934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:45.446677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'834","reported_seq":4329,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096720+0000","last_change":"2024-04-23T23:51:50.007789+0000","last_active":"2024-04-23T23:52:10.096720+0000","last_peered":"2024-04-23T23:52:10.096720+0000","last_clean":"2024-04-23T23:52:10.096720+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:10.096720+0000","last_undegraded":"2024-04-23T23:52:10.096720+0000","last_fullsized":"2024-04-23T23:52:10.096720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:44:24.111886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"119'379","reported_seq":3308,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382998+0000","last_change":"2024-04-23T23:51:50.014697+0000","last_active":"2024-04-23T23:52:09.382998+0000","last_peered":"2024-04-23T23:52:09.382998+0000","last_clean":"2024-04-23T23:52:09.382998+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:09.382998+0000","last_undegraded":"2024-04-23T23:52:09.382998+0000","last_fullsized":"2024-04-23T23:52:09.382998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:40.316577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"131'741","reported_seq":6498,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.383104+0000","last_change":"2024-04-23T23:51:50.013253+0000","last_active":"2024-04-23T23:52:09.383104+0000","last_peered":"2024-04-23T23:52:09.383104+0000","last_clean":"2024-04-23T23:52:09.383104+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:09.383104+0000","last_undegraded":"2024-04-23T23:52:09.383104+0000","last_fullsized":"2024-04-23T23:52:09.383104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:47.200572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"129'304","reported_seq":4086,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107808+0000","last_change":"2024-04-23T23:51:51.017335+0000","last_active":"2024-04-23T23:52:09.107808+0000","last_peered":"2024-04-23T23:52:09.107808+0000","last_clean":"2024-04-23T23:52:09.107808+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:09.107808+0000","last_undegraded":"2024-04-23T23:52:09.107808+0000","last_fullsized":"2024-04-23T23:52:09.107808+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:52:41.586532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"123'333","reported_seq":4979,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092378+0000","last_change":"2024-04-23T23:51:57.043361+0000","last_active":"2024-04-23T23:52:07.092378+0000","last_peered":"2024-04-23T23:52:07.092378+0000","last_clean":"2024-04-23T23:52:07.092378+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:07.092378+0000","last_undegraded":"2024-04-23T23:52:07.092378+0000","last_fullsized":"2024-04-23T23:52:07.092378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:06.039919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"119'788","reported_seq":6488,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107545+0000","last_change":"2024-04-23T23:51:57.044986+0000","last_active":"2024-04-23T23:52:09.107545+0000","last_peered":"2024-04-23T23:52:09.107545+0000","last_clean":"2024-04-23T23:52:09.107545+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:09.107545+0000","last_undegraded":"2024-04-23T23:52:09.107545+0000","last_fullsized":"2024-04-23T23:52:09.107545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:22.571801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"126'626","reported_seq":3564,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097250+0000","last_change":"2024-04-23T23:51:51.014869+0000","last_active":"2024-04-23T23:52:10.097250+0000","last_peered":"2024-04-23T23:52:10.097250+0000","last_clean":"2024-04-23T23:52:10.097250+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:10.097250+0000","last_undegraded":"2024-04-23T23:52:10.097250+0000","last_fullsized":"2024-04-23T23:52:10.097250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:25.995036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"131'620","reported_seq":6367,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107990+0000","last_change":"2024-04-23T23:51:51.021741+0000","last_active":"2024-04-23T23:52:09.107990+0000","last_peered":"2024-04-23T23:52:09.107990+0000","last_clean":"2024-04-23T23:52:09.107990+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:09.107990+0000","last_undegraded":"2024-04-23T23:52:09.107990+0000","last_fullsized":"2024-04-23T23:52:09.107990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:28:57.626808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"131'468","reported_seq":6523,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382726+0000","last_change":"2024-04-23T23:51:50.012988+0000","last_active":"2024-04-23T23:52:09.382726+0000","last_peered":"2024-04-23T23:52:09.382726+0000","last_clean":"2024-04-23T23:52:09.382726+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:09.382726+0000","last_undegraded":"2024-04-23T23:52:09.382726+0000","last_fullsized":"2024-04-23T23:52:09.382726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:48.845107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"129'366","reported_seq":2162,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844525+0000","last_change":"2024-04-23T23:51:50.011281+0000","last_active":"2024-04-23T23:52:07.844525+0000","last_peered":"2024-04-23T23:52:07.844525+0000","last_clean":"2024-04-23T23:52:07.844525+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:07.844525+0000","last_undegraded":"2024-04-23T23:52:07.844525+0000","last_fullsized":"2024-04-23T23:52:07.844525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:11.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"125'277","reported_seq":4560,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108088+0000","last_change":"2024-04-23T23:51:57.042203+0000","last_active":"2024-04-23T23:52:09.108088+0000","last_peered":"2024-04-23T23:52:09.108088+0000","last_clean":"2024-04-23T23:52:09.108088+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:09.108088+0000","last_undegraded":"2024-04-23T23:52:09.108088+0000","last_fullsized":"2024-04-23T23:52:09.108088+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:19.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105223+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:52:09.105223+0000","last_peered":"2024-04-23T23:52:09.105223+0000","last_clean":"2024-04-23T23:52:09.105223+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:09.105223+0000","last_undegraded":"2024-04-23T23:52:09.105223+0000","last_fullsized":"2024-04-23T23:52:09.105223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"131'864","reported_seq":3092,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107941+0000","last_change":"2024-04-23T23:51:57.042767+0000","last_active":"2024-04-23T23:52:09.107941+0000","last_peered":"2024-04-23T23:52:09.107941+0000","last_clean":"2024-04-23T23:52:09.107941+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:09.107941+0000","last_undegraded":"2024-04-23T23:52:09.107941+0000","last_fullsized":"2024-04-23T23:52:09.107941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:28:58.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107742+0000","last_change":"2024-04-23T23:51:53.964555+0000","last_active":"2024-04-23T23:52:09.107742+0000","last_peered":"2024-04-23T23:52:09.107742+0000","last_clean":"2024-04-23T23:52:09.107742+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:09.107742+0000","last_undegraded":"2024-04-23T23:52:09.107742+0000","last_fullsized":"2024-04-23T23:52:09.107742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:05:37.770013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'624","reported_seq":4639,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096820+0000","last_change":"2024-04-23T23:51:50.009306+0000","last_active":"2024-04-23T23:52:10.096820+0000","last_peered":"2024-04-23T23:52:10.096820+0000","last_clean":"2024-04-23T23:52:10.096820+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:10.096820+0000","last_undegraded":"2024-04-23T23:52:10.096820+0000","last_fullsized":"2024-04-23T23:52:10.096820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:34.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":272,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844652+0000","last_change":"2024-04-23T23:51:47.967029+0000","last_active":"2024-04-23T23:52:07.844652+0000","last_peered":"2024-04-23T23:52:07.844652+0000","last_clean":"2024-04-23T23:52:07.844652+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:07.844652+0000","last_undegraded":"2024-04-23T23:52:07.844652+0000","last_fullsized":"2024-04-23T23:52:07.844652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:28.475010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'460","reported_seq":2674,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107932+0000","last_change":"2024-04-23T23:51:57.043327+0000","last_active":"2024-04-23T23:52:09.107932+0000","last_peered":"2024-04-23T23:52:09.107932+0000","last_clean":"2024-04-23T23:52:09.107932+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:09.107932+0000","last_undegraded":"2024-04-23T23:52:09.107932+0000","last_fullsized":"2024-04-23T23:52:09.107932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:35.662689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108102+0000","last_change":"2024-04-23T23:51:53.964521+0000","last_active":"2024-04-23T23:52:09.108102+0000","last_peered":"2024-04-23T23:52:09.108102+0000","last_clean":"2024-04-23T23:52:09.108102+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:09.108102+0000","last_undegraded":"2024-04-23T23:52:09.108102+0000","last_fullsized":"2024-04-23T23:52:09.108102+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:14:20.619635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'541","reported_seq":9060,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382900+0000","last_change":"2024-04-23T23:51:50.014733+0000","last_active":"2024-04-23T23:52:09.382900+0000","last_peered":"2024-04-23T23:52:09.382900+0000","last_clean":"2024-04-23T23:52:09.382900+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:09.382900+0000","last_undegraded":"2024-04-23T23:52:09.382900+0000","last_fullsized":"2024-04-23T23:52:09.382900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:07.885750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"131'742","reported_seq":5032,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092516+0000","last_change":"2024-04-23T23:51:51.016283+0000","last_active":"2024-04-23T23:52:07.092516+0000","last_peered":"2024-04-23T23:52:07.092516+0000","last_clean":"2024-04-23T23:52:07.092516+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:07.092516+0000","last_undegraded":"2024-04-23T23:52:07.092516+0000","last_fullsized":"2024-04-23T23:52:07.092516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:19.673527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382865+0000","last_change":"2024-04-23T23:51:53.965136+0000","last_active":"2024-04-23T23:52:09.382865+0000","last_peered":"2024-04-23T23:52:09.382865+0000","last_clean":"2024-04-23T23:52:09.382865+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:09.382865+0000","last_undegraded":"2024-04-23T23:52:09.382865+0000","last_fullsized":"2024-04-23T23:52:09.382865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:40.481944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"126'317","reported_seq":2745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107507+0000","last_change":"2024-04-23T23:51:57.045408+0000","last_active":"2024-04-23T23:52:09.107507+0000","last_peered":"2024-04-23T23:52:09.107507+0000","last_clean":"2024-04-23T23:52:09.107507+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:09.107507+0000","last_undegraded":"2024-04-23T23:52:09.107507+0000","last_fullsized":"2024-04-23T23:52:09.107507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:34.567554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":271,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.547959+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:52:09.547959+0000","last_peered":"2024-04-23T23:52:09.547959+0000","last_clean":"2024-04-23T23:52:09.547959+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:09.547959+0000","last_undegraded":"2024-04-23T23:52:09.547959+0000","last_fullsized":"2024-04-23T23:52:09.547959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:26.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'353","reported_seq":3179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107819+0000","last_change":"2024-04-23T23:51:51.012712+0000","last_active":"2024-04-23T23:52:09.107819+0000","last_peered":"2024-04-23T23:52:09.107819+0000","last_clean":"2024-04-23T23:52:09.107819+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:09.107819+0000","last_undegraded":"2024-04-23T23:52:09.107819+0000","last_fullsized":"2024-04-23T23:52:09.107819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:52.621838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":282,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844466+0000","last_change":"2024-04-23T23:51:53.964436+0000","last_active":"2024-04-23T23:52:07.844466+0000","last_peered":"2024-04-23T23:52:07.844466+0000","last_clean":"2024-04-23T23:52:07.844466+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:07.844466+0000","last_undegraded":"2024-04-23T23:52:07.844466+0000","last_fullsized":"2024-04-23T23:52:07.844466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:59.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"129'630","reported_seq":2024,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096571+0000","last_change":"2024-04-23T23:51:50.007306+0000","last_active":"2024-04-23T23:52:10.096571+0000","last_peered":"2024-04-23T23:52:10.096571+0000","last_clean":"2024-04-23T23:52:10.096571+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:10.096571+0000","last_undegraded":"2024-04-23T23:52:10.096571+0000","last_fullsized":"2024-04-23T23:52:10.096571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:42.222266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382664+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:52:09.382664+0000","last_peered":"2024-04-23T23:52:09.382664+0000","last_clean":"2024-04-23T23:52:09.382664+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:09.382664+0000","last_undegraded":"2024-04-23T23:52:09.382664+0000","last_fullsized":"2024-04-23T23:52:09.382664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":444,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844531+0000","last_change":"2024-04-23T23:51:51.009383+0000","last_active":"2024-04-23T23:52:07.844531+0000","last_peered":"2024-04-23T23:52:07.844531+0000","last_clean":"2024-04-23T23:52:07.844531+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:07.844531+0000","last_undegraded":"2024-04-23T23:52:07.844531+0000","last_fullsized":"2024-04-23T23:52:07.844531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:07.935657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"125'321","reported_seq":1922,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096768+0000","last_change":"2024-04-23T23:51:57.041481+0000","last_active":"2024-04-23T23:52:10.096768+0000","last_peered":"2024-04-23T23:52:10.096768+0000","last_clean":"2024-04-23T23:52:10.096768+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:10.096768+0000","last_undegraded":"2024-04-23T23:52:10.096768+0000","last_fullsized":"2024-04-23T23:52:10.096768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:07.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'516","reported_seq":2892,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.092470+0000","last_change":"2024-04-23T23:51:50.014340+0000","last_active":"2024-04-23T23:52:07.092470+0000","last_peered":"2024-04-23T23:52:07.092470+0000","last_clean":"2024-04-23T23:52:07.092470+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:07.092470+0000","last_undegraded":"2024-04-23T23:52:07.092470+0000","last_fullsized":"2024-04-23T23:52:07.092470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:49:40.738996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"129'442","reported_seq":3505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:50.008342+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:00.995303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'443","reported_seq":2952,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097048+0000","last_change":"2024-04-23T23:51:51.016535+0000","last_active":"2024-04-23T23:52:10.097048+0000","last_peered":"2024-04-23T23:52:10.097048+0000","last_clean":"2024-04-23T23:52:10.097048+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:10.097048+0000","last_undegraded":"2024-04-23T23:52:10.097048+0000","last_fullsized":"2024-04-23T23:52:10.097048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:47:29.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'443","reported_seq":4073,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382915+0000","last_change":"2024-04-23T23:51:50.014803+0000","last_active":"2024-04-23T23:52:09.382915+0000","last_peered":"2024-04-23T23:52:09.382915+0000","last_clean":"2024-04-23T23:52:09.382915+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:09.382915+0000","last_undegraded":"2024-04-23T23:52:09.382915+0000","last_fullsized":"2024-04-23T23:52:09.382915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:47.080894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'363","reported_seq":3494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844572+0000","last_change":"2024-04-23T23:51:50.012198+0000","last_active":"2024-04-23T23:52:07.844572+0000","last_peered":"2024-04-23T23:52:07.844572+0000","last_clean":"2024-04-23T23:52:07.844572+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:07.844572+0000","last_undegraded":"2024-04-23T23:52:07.844572+0000","last_fullsized":"2024-04-23T23:52:07.844572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.339718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"123'586","reported_seq":3674,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844493+0000","last_change":"2024-04-23T23:51:50.010749+0000","last_active":"2024-04-23T23:52:07.844493+0000","last_peered":"2024-04-23T23:52:07.844493+0000","last_clean":"2024-04-23T23:52:07.844493+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:07.844493+0000","last_undegraded":"2024-04-23T23:52:07.844493+0000","last_fullsized":"2024-04-23T23:52:07.844493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:23.648951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"131'858","reported_seq":4029,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844441+0000","last_change":"2024-04-23T23:51:50.010902+0000","last_active":"2024-04-23T23:52:07.844441+0000","last_peered":"2024-04-23T23:52:07.844441+0000","last_clean":"2024-04-23T23:52:07.844441+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:07.844441+0000","last_undegraded":"2024-04-23T23:52:07.844441+0000","last_fullsized":"2024-04-23T23:52:07.844441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:29.439340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"129'321","reported_seq":5844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:51.012948+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:42.912702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"118'411","reported_seq":4195,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844358+0000","last_change":"2024-04-23T23:51:50.007735+0000","last_active":"2024-04-23T23:52:07.844358+0000","last_peered":"2024-04-23T23:52:07.844358+0000","last_clean":"2024-04-23T23:52:07.844358+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:07.844358+0000","last_undegraded":"2024-04-23T23:52:07.844358+0000","last_fullsized":"2024-04-23T23:52:07.844358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:10:47.430347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"126'451","reported_seq":2347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382665+0000","last_change":"2024-04-23T23:51:50.011706+0000","last_active":"2024-04-23T23:52:09.382665+0000","last_peered":"2024-04-23T23:52:09.382665+0000","last_clean":"2024-04-23T23:52:09.382665+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:09.382665+0000","last_undegraded":"2024-04-23T23:52:09.382665+0000","last_fullsized":"2024-04-23T23:52:09.382665+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:50.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"126'486","reported_seq":3064,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-23T23:52:07.844367+0000","last_change":"2024-04-23T23:51:50.011755+0000","last_active":"2024-04-23T23:52:07.844367+0000","last_peered":"2024-04-23T23:52:07.844367+0000","last_clean":"2024-04-23T23:52:07.844367+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:07.844367+0000","last_undegraded":"2024-04-23T23:52:07.844367+0000","last_fullsized":"2024-04-23T23:52:07.844367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:45:45.614364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16569,"ondisk_log_size":16569,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168598,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46924,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324916,"statfs":{"total":96636764160,"available":96588713984,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2338816,"data_stored":1427202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50936,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320904,"statfs":{"total":96636764160,"available":96584605696,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":836853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1744896,"data_stored":836834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:15.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T23:52:15.471 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:15.471 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":148,"fsid":"6a51c05a-822c-4999-80c7-e8408cd2a202","created":"2024-04-23T23:49:23.155452+0000","modified":"2024-04-23T23:52:09.101343+0000","last_up_change":"2024-04-23T23:50:32.112122+0000","last_in_change":"2024-04-23T23:51:52.954149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T23:49:35.387984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T23:49:41.909447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-04-23T23:49:49.482932+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a84eb3e-7d8a-4436-956d-f7da68d06a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6820","nonce":3364976130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6824","nonce":3364976130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6831","nonce":3364976130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3364976130},{"type":"v1","addr":"172.21.15.92:6828","nonce":3364976130}]},"public_addr":"172.21.15.92:6820/3364976130","cluster_addr":"172.21.15.92:6824/3364976130","heartbeat_back_addr":"172.21.15.92:6831/3364976130","heartbeat_front_addr":"172.21.15.92:6828/3364976130","state":["exists","up"]},{"osd":1,"uuid":"be35c480-9a84-4846-9f0b-043c7c2c9f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6817","nonce":2924722109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6821","nonce":2924722109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6829","nonce":2924722109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6823","nonce":2924722109},{"type":"v1","addr":"172.21.15.92:6825","nonce":2924722109}]},"public_addr":"172.21.15.92:6817/2924722109","cluster_addr":"172.21.15.92:6821/2924722109","heartbeat_back_addr":"172.21.15.92:6829/2924722109","heartbeat_front_addr":"172.21.15.92:6825/2924722109","state":["exists","up"]},{"osd":2,"uuid":"64e8bbbb-690a-4267-a030-30ec66f30bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6801","nonce":3952319845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6803","nonce":3952319845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6807","nonce":3952319845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3952319845},{"type":"v1","addr":"172.21.15.92:6805","nonce":3952319845}]},"public_addr":"172.21.15.92:6801/3952319845","cluster_addr":"172.21.15.92:6803/3952319845","heartbeat_back_addr":"172.21.15.92:6807/3952319845","heartbeat_front_addr":"172.21.15.92:6805/3952319845","state":["exists","up"]},{"osd":3,"uuid":"692beda9-1302-4d96-81ab-62c40a12e3f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":38,"up_from":56,"up_thru":135,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6809","nonce":1499077312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6811","nonce":1499077312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6815","nonce":1499077312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1499077312},{"type":"v1","addr":"172.21.15.92:6813","nonce":1499077312}]},"public_addr":"172.21.15.92:6809/1499077312","cluster_addr":"172.21.15.92:6811/1499077312","heartbeat_back_addr":"172.21.15.92:6815/1499077312","heartbeat_front_addr":"172.21.15.92:6813/1499077312","state":["exists","up"]},{"osd":4,"uuid":"705f6b1f-c35d-4ed8-a2c9-70b9324691e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6825","nonce":1281396523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6827","nonce":1281396523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6831","nonce":1281396523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1281396523},{"type":"v1","addr":"172.21.15.119:6829","nonce":1281396523}]},"public_addr":"172.21.15.119:6825/1281396523","cluster_addr":"172.21.15.119:6827/1281396523","heartbeat_back_addr":"172.21.15.119:6831/1281396523","heartbeat_front_addr":"172.21.15.119:6829/1281396523","state":["exists","up"]},{"osd":5,"uuid":"eb37446d-b5c9-4200-a1e9-38c1f6268401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6801","nonce":612886539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6803","nonce":612886539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6807","nonce":612886539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":612886539},{"type":"v1","addr":"172.21.15.119:6805","nonce":612886539}]},"public_addr":"172.21.15.119:6801/612886539","cluster_addr":"172.21.15.119:6803/612886539","heartbeat_back_addr":"172.21.15.119:6807/612886539","heartbeat_front_addr":"172.21.15.119:6805/612886539","state":["exists","up"]},{"osd":6,"uuid":"c38a4da5-1017-4bb9-86ab-999ea57999c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6817","nonce":2243867625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6819","nonce":2243867625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6823","nonce":2243867625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2243867625},{"type":"v1","addr":"172.21.15.119:6821","nonce":2243867625}]},"public_addr":"172.21.15.119:6817/2243867625","cluster_addr":"172.21.15.119:6819/2243867625","heartbeat_back_addr":"172.21.15.119:6823/2243867625","heartbeat_front_addr":"172.21.15.119:6821/2243867625","state":["exists","up"]},{"osd":7,"uuid":"a921d465-920a-4ad0-839f-d97cd5e6ce62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6809","nonce":3209176445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6811","nonce":3209176445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6815","nonce":3209176445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3209176445},{"type":"v1","addr":"172.21.15.119:6813","nonce":3209176445}]},"public_addr":"172.21.15.119:6809/3209176445","cluster_addr":"172.21.15.119:6811/3209176445","heartbeat_back_addr":"172.21.15.119:6815/3209176445","heartbeat_front_addr":"172.21.15.119:6813/3209176445","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.405502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.334094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.268468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.367807+0000","dead_epoch":39},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.328752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.255936+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.298943+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T23:49:31.352632+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[5,3]},{"pgid":"3.14","osds":[1,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T23:52:16.484 INFO:tasks.ceph:Scrubbing osd.0 2024-04-23T23:52:16.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:16.622 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:16.622 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:16.622 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:16.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-23T23:52:16.920 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T23:52:16.933 INFO:tasks.ceph:Scrubbing osd.1 2024-04-23T23:52:16.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:17.054 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:17.054 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:17.054 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:17.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-23T23:52:17.349 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T23:52:17.362 INFO:tasks.ceph:Scrubbing osd.2 2024-04-23T23:52:17.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:17.484 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:17.484 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:17.485 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:17.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-23T23:52:17.777 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T23:52:17.790 INFO:tasks.ceph:Scrubbing osd.3 2024-04-23T23:52:17.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:17.911 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:17.911 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:17.911 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:17.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-23T23:52:18.208 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2024-04-23T23:52:18.221 INFO:tasks.ceph:Scrubbing osd.4 2024-04-23T23:52:18.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:18.343 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:18.343 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:18.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-23T23:52:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2024-04-23T23:52:18.650 INFO:tasks.ceph:Scrubbing osd.5 2024-04-23T23:52:18.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:18.772 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:18.772 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:18.772 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:18.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-23T23:52:19.069 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2024-04-23T23:52:19.082 INFO:tasks.ceph:Scrubbing osd.6 2024-04-23T23:52:19.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:19.205 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:19.205 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:19.205 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:19.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-23T23:52:19.498 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2024-04-23T23:52:19.511 INFO:tasks.ceph:Scrubbing osd.7 2024-04-23T23:52:19.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-23T23:52:19.633 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-23T23:52:19.633 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T23:52:19.633 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-23T23:52:19.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-23T23:52:19.929 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2024-04-23T23:52:19.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:20.229 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:20.241 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-04-23T23:52:19.384961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16686,"ondisk_log_size":16686,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389500,"kb_used_data":14992,"kb_used_omap":21,"kb_used_meta":374378,"kb_avail":754585220,"statfs":{"total":773094113280,"available":772695265280,"internally_reserved":0,"allocated":15351808,"data_stored":7969382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":383363445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.281061"},"pg_stats":[{"pgid":"3.1f","version":"131'661","reported_seq":8954,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107934+0000","last_change":"2024-04-23T23:51:51.018566+0000","last_active":"2024-04-23T23:52:09.107934+0000","last_peered":"2024-04-23T23:52:09.107934+0000","last_clean":"2024-04-23T23:52:09.107934+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:09.107934+0000","last_undegraded":"2024-04-23T23:52:09.107934+0000","last_fullsized":"2024-04-23T23:52:09.107934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:45.446677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'834","reported_seq":4329,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096720+0000","last_change":"2024-04-23T23:51:50.007789+0000","last_active":"2024-04-23T23:52:10.096720+0000","last_peered":"2024-04-23T23:52:10.096720+0000","last_clean":"2024-04-23T23:52:10.096720+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:10.096720+0000","last_undegraded":"2024-04-23T23:52:10.096720+0000","last_fullsized":"2024-04-23T23:52:10.096720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:44:24.111886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"119'379","reported_seq":3308,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382998+0000","last_change":"2024-04-23T23:51:50.014697+0000","last_active":"2024-04-23T23:52:09.382998+0000","last_peered":"2024-04-23T23:52:09.382998+0000","last_clean":"2024-04-23T23:52:09.382998+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:09.382998+0000","last_undegraded":"2024-04-23T23:52:09.382998+0000","last_fullsized":"2024-04-23T23:52:09.382998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:40.316577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"131'741","reported_seq":6498,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.383104+0000","last_change":"2024-04-23T23:51:50.013253+0000","last_active":"2024-04-23T23:52:09.383104+0000","last_peered":"2024-04-23T23:52:09.383104+0000","last_clean":"2024-04-23T23:52:09.383104+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:09.383104+0000","last_undegraded":"2024-04-23T23:52:09.383104+0000","last_fullsized":"2024-04-23T23:52:09.383104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:47.200572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"129'304","reported_seq":4086,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107808+0000","last_change":"2024-04-23T23:51:51.017335+0000","last_active":"2024-04-23T23:52:09.107808+0000","last_peered":"2024-04-23T23:52:09.107808+0000","last_clean":"2024-04-23T23:52:09.107808+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:09.107808+0000","last_undegraded":"2024-04-23T23:52:09.107808+0000","last_fullsized":"2024-04-23T23:52:09.107808+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:52:41.586532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"123'333","reported_seq":4982,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097160+0000","last_change":"2024-04-23T23:51:57.043361+0000","last_active":"2024-04-23T23:52:10.097160+0000","last_peered":"2024-04-23T23:52:10.097160+0000","last_clean":"2024-04-23T23:52:10.097160+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:10.097160+0000","last_undegraded":"2024-04-23T23:52:10.097160+0000","last_fullsized":"2024-04-23T23:52:10.097160+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:06.039919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"119'788","reported_seq":6488,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107545+0000","last_change":"2024-04-23T23:51:57.044986+0000","last_active":"2024-04-23T23:52:09.107545+0000","last_peered":"2024-04-23T23:52:09.107545+0000","last_clean":"2024-04-23T23:52:09.107545+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:09.107545+0000","last_undegraded":"2024-04-23T23:52:09.107545+0000","last_fullsized":"2024-04-23T23:52:09.107545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:22.571801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"126'626","reported_seq":3564,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097250+0000","last_change":"2024-04-23T23:51:51.014869+0000","last_active":"2024-04-23T23:52:10.097250+0000","last_peered":"2024-04-23T23:52:10.097250+0000","last_clean":"2024-04-23T23:52:10.097250+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:10.097250+0000","last_undegraded":"2024-04-23T23:52:10.097250+0000","last_fullsized":"2024-04-23T23:52:10.097250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:25.995036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"131'620","reported_seq":6367,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107990+0000","last_change":"2024-04-23T23:51:51.021741+0000","last_active":"2024-04-23T23:52:09.107990+0000","last_peered":"2024-04-23T23:52:09.107990+0000","last_clean":"2024-04-23T23:52:09.107990+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:09.107990+0000","last_undegraded":"2024-04-23T23:52:09.107990+0000","last_fullsized":"2024-04-23T23:52:09.107990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:28:57.626808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"131'468","reported_seq":6523,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382726+0000","last_change":"2024-04-23T23:51:50.012988+0000","last_active":"2024-04-23T23:52:09.382726+0000","last_peered":"2024-04-23T23:52:09.382726+0000","last_clean":"2024-04-23T23:52:09.382726+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:09.382726+0000","last_undegraded":"2024-04-23T23:52:09.382726+0000","last_fullsized":"2024-04-23T23:52:09.382726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:48.845107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"129'366","reported_seq":2165,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011434+0000","last_change":"2024-04-23T23:51:50.011281+0000","last_active":"2024-04-23T23:52:10.011434+0000","last_peered":"2024-04-23T23:52:10.011434+0000","last_clean":"2024-04-23T23:52:10.011434+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:10.011434+0000","last_undegraded":"2024-04-23T23:52:10.011434+0000","last_fullsized":"2024-04-23T23:52:10.011434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:31:11.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"125'277","reported_seq":4560,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108088+0000","last_change":"2024-04-23T23:51:57.042203+0000","last_active":"2024-04-23T23:52:09.108088+0000","last_peered":"2024-04-23T23:52:09.108088+0000","last_clean":"2024-04-23T23:52:09.108088+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:09.108088+0000","last_undegraded":"2024-04-23T23:52:09.108088+0000","last_fullsized":"2024-04-23T23:52:09.108088+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:19.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105223+0000","last_change":"2024-04-23T23:49:43.259954+0000","last_active":"2024-04-23T23:52:09.105223+0000","last_peered":"2024-04-23T23:52:09.105223+0000","last_clean":"2024-04-23T23:52:09.105223+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:09.105223+0000","last_undegraded":"2024-04-23T23:52:09.105223+0000","last_fullsized":"2024-04-23T23:52:09.105223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:34.679194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"131'864","reported_seq":3092,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107941+0000","last_change":"2024-04-23T23:51:57.042767+0000","last_active":"2024-04-23T23:52:09.107941+0000","last_peered":"2024-04-23T23:52:09.107941+0000","last_clean":"2024-04-23T23:52:09.107941+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:09.107941+0000","last_undegraded":"2024-04-23T23:52:09.107941+0000","last_fullsized":"2024-04-23T23:52:09.107941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:28:58.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107742+0000","last_change":"2024-04-23T23:51:53.964555+0000","last_active":"2024-04-23T23:52:09.107742+0000","last_peered":"2024-04-23T23:52:09.107742+0000","last_clean":"2024-04-23T23:52:09.107742+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:09.107742+0000","last_undegraded":"2024-04-23T23:52:09.107742+0000","last_fullsized":"2024-04-23T23:52:09.107742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:05:37.770013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'624","reported_seq":4639,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096820+0000","last_change":"2024-04-23T23:51:50.009306+0000","last_active":"2024-04-23T23:52:10.096820+0000","last_peered":"2024-04-23T23:52:10.096820+0000","last_clean":"2024-04-23T23:52:10.096820+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:10.096820+0000","last_undegraded":"2024-04-23T23:52:10.096820+0000","last_fullsized":"2024-04-23T23:52:10.096820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:15:34.910125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011664+0000","last_change":"2024-04-23T23:51:47.967029+0000","last_active":"2024-04-23T23:52:10.011664+0000","last_peered":"2024-04-23T23:52:10.011664+0000","last_clean":"2024-04-23T23:52:10.011664+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:10.011664+0000","last_undegraded":"2024-04-23T23:52:10.011664+0000","last_fullsized":"2024-04-23T23:52:10.011664+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:28.475010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'460","reported_seq":2674,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107932+0000","last_change":"2024-04-23T23:51:57.043327+0000","last_active":"2024-04-23T23:52:09.107932+0000","last_peered":"2024-04-23T23:52:09.107932+0000","last_clean":"2024-04-23T23:52:09.107932+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:09.107932+0000","last_undegraded":"2024-04-23T23:52:09.107932+0000","last_fullsized":"2024-04-23T23:52:09.107932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:35.662689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.108102+0000","last_change":"2024-04-23T23:51:53.964521+0000","last_active":"2024-04-23T23:52:09.108102+0000","last_peered":"2024-04-23T23:52:09.108102+0000","last_clean":"2024-04-23T23:52:09.108102+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:09.108102+0000","last_undegraded":"2024-04-23T23:52:09.108102+0000","last_fullsized":"2024-04-23T23:52:09.108102+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:14:20.619635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'541","reported_seq":9060,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382900+0000","last_change":"2024-04-23T23:51:50.014733+0000","last_active":"2024-04-23T23:52:09.382900+0000","last_peered":"2024-04-23T23:52:09.382900+0000","last_clean":"2024-04-23T23:52:09.382900+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:09.382900+0000","last_undegraded":"2024-04-23T23:52:09.382900+0000","last_fullsized":"2024-04-23T23:52:09.382900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:07.885750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"131'742","reported_seq":5035,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097273+0000","last_change":"2024-04-23T23:51:51.016283+0000","last_active":"2024-04-23T23:52:10.097273+0000","last_peered":"2024-04-23T23:52:10.097273+0000","last_clean":"2024-04-23T23:52:10.097273+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:10.097273+0000","last_undegraded":"2024-04-23T23:52:10.097273+0000","last_fullsized":"2024-04-23T23:52:10.097273+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:19.673527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382865+0000","last_change":"2024-04-23T23:51:53.965136+0000","last_active":"2024-04-23T23:52:09.382865+0000","last_peered":"2024-04-23T23:52:09.382865+0000","last_clean":"2024-04-23T23:52:09.382865+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:09.382865+0000","last_undegraded":"2024-04-23T23:52:09.382865+0000","last_fullsized":"2024-04-23T23:52:09.382865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:40.481944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"126'317","reported_seq":2745,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107507+0000","last_change":"2024-04-23T23:51:57.045408+0000","last_active":"2024-04-23T23:52:09.107507+0000","last_peered":"2024-04-23T23:52:09.107507+0000","last_clean":"2024-04-23T23:52:09.107507+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:09.107507+0000","last_undegraded":"2024-04-23T23:52:09.107507+0000","last_fullsized":"2024-04-23T23:52:09.107507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:34.567554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":272,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.548001+0000","last_change":"2024-04-23T23:50:32.953680+0000","last_active":"2024-04-23T23:52:09.548001+0000","last_peered":"2024-04-23T23:52:09.548001+0000","last_clean":"2024-04-23T23:52:09.548001+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:09.548001+0000","last_undegraded":"2024-04-23T23:52:09.548001+0000","last_fullsized":"2024-04-23T23:52:09.548001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'353","reported_seq":3179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.107819+0000","last_change":"2024-04-23T23:51:51.012712+0000","last_active":"2024-04-23T23:52:09.107819+0000","last_peered":"2024-04-23T23:52:09.107819+0000","last_clean":"2024-04-23T23:52:09.107819+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:09.107819+0000","last_undegraded":"2024-04-23T23:52:09.107819+0000","last_fullsized":"2024-04-23T23:52:09.107819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:52.621838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":285,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011391+0000","last_change":"2024-04-23T23:51:53.964436+0000","last_active":"2024-04-23T23:52:10.011391+0000","last_peered":"2024-04-23T23:52:10.011391+0000","last_clean":"2024-04-23T23:52:10.011391+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:10.011391+0000","last_undegraded":"2024-04-23T23:52:10.011391+0000","last_fullsized":"2024-04-23T23:52:10.011391+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:59.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"129'630","reported_seq":2024,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096571+0000","last_change":"2024-04-23T23:51:50.007306+0000","last_active":"2024-04-23T23:52:10.096571+0000","last_peered":"2024-04-23T23:52:10.096571+0000","last_clean":"2024-04-23T23:52:10.096571+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:10.096571+0000","last_undegraded":"2024-04-23T23:52:10.096571+0000","last_fullsized":"2024-04-23T23:52:10.096571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:32:42.222266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382664+0000","last_change":"2024-04-23T23:49:43.264487+0000","last_active":"2024-04-23T23:52:09.382664+0000","last_peered":"2024-04-23T23:52:09.382664+0000","last_clean":"2024-04-23T23:52:09.382664+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:09.382664+0000","last_undegraded":"2024-04-23T23:52:09.382664+0000","last_fullsized":"2024-04-23T23:52:09.382664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:42.246487+0000","last_clean_scrub_stamp":"2024-04-23T23:49:42.246487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:31:07.155410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":447,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011780+0000","last_change":"2024-04-23T23:51:51.009383+0000","last_active":"2024-04-23T23:52:10.011780+0000","last_peered":"2024-04-23T23:52:10.011780+0000","last_clean":"2024-04-23T23:52:10.011780+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:10.011780+0000","last_undegraded":"2024-04-23T23:52:10.011780+0000","last_fullsized":"2024-04-23T23:52:10.011780+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:36.181026+0000","last_clean_scrub_stamp":"2024-04-23T23:49:36.181026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:07.935657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"125'321","reported_seq":1922,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.096768+0000","last_change":"2024-04-23T23:51:57.041481+0000","last_active":"2024-04-23T23:52:10.096768+0000","last_peered":"2024-04-23T23:52:10.096768+0000","last_clean":"2024-04-23T23:52:10.096768+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:10.096768+0000","last_undegraded":"2024-04-23T23:52:10.096768+0000","last_fullsized":"2024-04-23T23:52:10.096768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:07.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'516","reported_seq":2895,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097191+0000","last_change":"2024-04-23T23:51:50.014340+0000","last_active":"2024-04-23T23:52:10.097191+0000","last_peered":"2024-04-23T23:52:10.097191+0000","last_clean":"2024-04-23T23:52:10.097191+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:10.097191+0000","last_undegraded":"2024-04-23T23:52:10.097191+0000","last_fullsized":"2024-04-23T23:52:10.097191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:49:40.738996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"129'442","reported_seq":3505,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105181+0000","last_change":"2024-04-23T23:51:50.008342+0000","last_active":"2024-04-23T23:52:09.105181+0000","last_peered":"2024-04-23T23:52:09.105181+0000","last_clean":"2024-04-23T23:52:09.105181+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:09.105181+0000","last_undegraded":"2024-04-23T23:52:09.105181+0000","last_fullsized":"2024-04-23T23:52:09.105181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:00.995303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'443","reported_seq":2952,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.097048+0000","last_change":"2024-04-23T23:51:51.016535+0000","last_active":"2024-04-23T23:52:10.097048+0000","last_peered":"2024-04-23T23:52:10.097048+0000","last_clean":"2024-04-23T23:52:10.097048+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:10.097048+0000","last_undegraded":"2024-04-23T23:52:10.097048+0000","last_fullsized":"2024-04-23T23:52:10.097048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:47:29.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'443","reported_seq":4073,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382915+0000","last_change":"2024-04-23T23:51:50.014803+0000","last_active":"2024-04-23T23:52:09.382915+0000","last_peered":"2024-04-23T23:52:09.382915+0000","last_clean":"2024-04-23T23:52:09.382915+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:09.382915+0000","last_undegraded":"2024-04-23T23:52:09.382915+0000","last_fullsized":"2024-04-23T23:52:09.382915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:47.080894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'363","reported_seq":3497,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011629+0000","last_change":"2024-04-23T23:51:50.012198+0000","last_active":"2024-04-23T23:52:10.011629+0000","last_peered":"2024-04-23T23:52:10.011629+0000","last_clean":"2024-04-23T23:52:10.011629+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:10.011629+0000","last_undegraded":"2024-04-23T23:52:10.011629+0000","last_fullsized":"2024-04-23T23:52:10.011629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.339718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"123'586","reported_seq":3677,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011596+0000","last_change":"2024-04-23T23:51:50.010749+0000","last_active":"2024-04-23T23:52:10.011596+0000","last_peered":"2024-04-23T23:52:10.011596+0000","last_clean":"2024-04-23T23:52:10.011596+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:10.011596+0000","last_undegraded":"2024-04-23T23:52:10.011596+0000","last_fullsized":"2024-04-23T23:52:10.011596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:23.648951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"131'858","reported_seq":4032,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011670+0000","last_change":"2024-04-23T23:51:50.010902+0000","last_active":"2024-04-23T23:52:10.011670+0000","last_peered":"2024-04-23T23:52:10.011670+0000","last_clean":"2024-04-23T23:52:10.011670+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:10.011670+0000","last_undegraded":"2024-04-23T23:52:10.011670+0000","last_fullsized":"2024-04-23T23:52:10.011670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:42:29.439340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"129'321","reported_seq":5845,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.105252+0000","last_change":"2024-04-23T23:51:51.012948+0000","last_active":"2024-04-23T23:52:09.105252+0000","last_peered":"2024-04-23T23:52:09.105252+0000","last_clean":"2024-04-23T23:52:09.105252+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:09.105252+0000","last_undegraded":"2024-04-23T23:52:09.105252+0000","last_fullsized":"2024-04-23T23:52:09.105252+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:42.912702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"118'411","reported_seq":4199,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011626+0000","last_change":"2024-04-23T23:51:50.007735+0000","last_active":"2024-04-23T23:52:10.011626+0000","last_peered":"2024-04-23T23:52:10.011626+0000","last_clean":"2024-04-23T23:52:10.011626+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:10.011626+0000","last_undegraded":"2024-04-23T23:52:10.011626+0000","last_fullsized":"2024-04-23T23:52:10.011626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:10:47.430347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"126'451","reported_seq":2347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:09.382665+0000","last_change":"2024-04-23T23:51:50.011706+0000","last_active":"2024-04-23T23:52:09.382665+0000","last_peered":"2024-04-23T23:52:09.382665+0000","last_clean":"2024-04-23T23:52:09.382665+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:09.382665+0000","last_undegraded":"2024-04-23T23:52:09.382665+0000","last_fullsized":"2024-04-23T23:52:09.382665+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:50.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"126'486","reported_seq":3068,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:10.011608+0000","last_change":"2024-04-23T23:51:50.011755+0000","last_active":"2024-04-23T23:52:10.011608+0000","last_peered":"2024-04-23T23:52:10.011608+0000","last_clean":"2024-04-23T23:52:10.011608+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:10.011608+0000","last_undegraded":"2024-04-23T23:52:10.011608+0000","last_fullsized":"2024-04-23T23:52:10.011608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:49:50.299041+0000","last_clean_scrub_stamp":"2024-04-23T23:49:50.299041+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:45:45.614364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16569,"ondisk_log_size":16569,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168599,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46924,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324916,"statfs":{"total":96636764160,"available":96588713984,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51536,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320304,"statfs":{"total":96636764160,"available":96583991296,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50956,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320884,"statfs":{"total":96636764160,"available":96584585216,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55120,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316720,"statfs":{"total":96636764160,"available":96580321280,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:20.246 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.246 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.246 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.246 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.246 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.246 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.247 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.248 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T23:49:42.246487+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=42, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T23:49:36.181026+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=36, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.249 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-23T23:49:50.299041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=49, tm_sec=50, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=23, tm_min=52, tm_sec=15, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T23:52:20.250 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T23:52:40.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T23:52:40.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-23T23:52:40.551 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-23T23:52:40.562 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-04-23T23:52:39.389736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":403,"num_read":156559,"num_read_kb":614308,"num_write":17810,"num_write_kb":305539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3129,"num_bytes_recovered":29162139,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17094,"ondisk_log_size":17094,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":397776,"kb_used_data":14992,"kb_used_omap":21,"kb_used_meta":382698,"kb_avail":754576944,"statfs":{"total":773094113280,"available":772686790656,"internally_reserved":0,"allocated":15351808,"data_stored":7969382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22155,"internal_metadata":391883125},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002969"},"pg_stats":[{"pgid":"3.1f","version":"148'676","reported_seq":8978,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.773230+0000","last_change":"2024-04-23T23:52:20.773265+0000","last_active":"2024-04-23T23:52:20.773230+0000","last_peered":"2024-04-23T23:52:20.773230+0000","last_clean":"2024-04-23T23:52:20.773230+0000","last_became_active":"2024-04-23T23:51:51.013710+0000","last_became_peered":"2024-04-23T23:51:51.013710+0000","last_unstale":"2024-04-23T23:52:20.773230+0000","last_undegraded":"2024-04-23T23:52:20.773230+0000","last_fullsized":"2024-04-23T23:52:20.773230+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'676","last_scrub_stamp":"2024-04-23T23:52:20.773158+0000","last_deep_scrub":"148'676","last_deep_scrub_stamp":"2024-04-23T23:52:20.773158+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.773158+0000","objects_scrubbed":15,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:10:02.057325+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0032729719999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":10084,"num_read_kb":23884,"num_write":791,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":80,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"148'842","reported_seq":4347,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.260027+0000","last_change":"2024-04-23T23:52:20.260027+0000","last_active":"2024-04-23T23:52:20.260027+0000","last_peered":"2024-04-23T23:52:20.260027+0000","last_clean":"2024-04-23T23:52:20.260027+0000","last_became_active":"2024-04-23T23:50:36.142035+0000","last_became_peered":"2024-04-23T23:50:36.142035+0000","last_unstale":"2024-04-23T23:52:20.260027+0000","last_undegraded":"2024-04-23T23:52:20.260027+0000","last_fullsized":"2024-04-23T23:52:20.260027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'842","last_scrub_stamp":"2024-04-23T23:52:20.259910+0000","last_deep_scrub":"148'842","last_deep_scrub_stamp":"2024-04-23T23:52:20.259910+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.259910+0000","objects_scrubbed":8,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:14.162903+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.001920452,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5833,"num_read_kb":12899,"num_write":868,"num_write_kb":8207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":623270,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"148'391","reported_seq":3330,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.445096+0000","last_change":"2024-04-23T23:52:20.445096+0000","last_active":"2024-04-23T23:52:20.445096+0000","last_peered":"2024-04-23T23:52:20.445096+0000","last_clean":"2024-04-23T23:52:20.445096+0000","last_became_active":"2024-04-23T23:49:51.317761+0000","last_became_peered":"2024-04-23T23:49:51.317761+0000","last_unstale":"2024-04-23T23:52:20.445096+0000","last_undegraded":"2024-04-23T23:52:20.445096+0000","last_fullsized":"2024-04-23T23:52:20.445096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'391","last_scrub_stamp":"2024-04-23T23:52:20.444907+0000","last_deep_scrub":"148'391","last_deep_scrub_stamp":"2024-04-23T23:52:20.444907+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.444907+0000","objects_scrubbed":12,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:10.957854+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0032888069999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3077,"num_read_kb":15772,"num_write":379,"num_write_kb":4684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"148'752","reported_seq":6519,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:24.488395+0000","last_change":"2024-04-23T23:52:24.488447+0000","last_active":"2024-04-23T23:52:24.488395+0000","last_peered":"2024-04-23T23:52:24.488395+0000","last_clean":"2024-04-23T23:52:24.488395+0000","last_became_active":"2024-04-23T23:51:32.054738+0000","last_became_peered":"2024-04-23T23:51:32.054738+0000","last_unstale":"2024-04-23T23:52:24.488395+0000","last_undegraded":"2024-04-23T23:52:24.488395+0000","last_fullsized":"2024-04-23T23:52:24.488395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'752","last_scrub_stamp":"2024-04-23T23:52:24.488283+0000","last_deep_scrub":"148'752","last_deep_scrub_stamp":"2024-04-23T23:52:24.488283+0000","last_clean_scrub_stamp":"2024-04-23T23:52:24.488283+0000","objects_scrubbed":11,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:47:30.359767+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0027387620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7331,"num_read_kb":30564,"num_write":899,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":4468964,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'320","reported_seq":4111,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.750164+0000","last_change":"2024-04-23T23:52:19.750200+0000","last_active":"2024-04-23T23:52:19.750164+0000","last_peered":"2024-04-23T23:52:19.750164+0000","last_clean":"2024-04-23T23:52:19.750164+0000","last_became_active":"2024-04-23T23:51:51.011907+0000","last_became_peered":"2024-04-23T23:51:51.011907+0000","last_unstale":"2024-04-23T23:52:19.750164+0000","last_undegraded":"2024-04-23T23:52:19.750164+0000","last_fullsized":"2024-04-23T23:52:19.750164+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'320","last_scrub_stamp":"2024-04-23T23:52:19.750097+0000","last_deep_scrub":"148'320","last_deep_scrub_stamp":"2024-04-23T23:52:19.750097+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.750097+0000","objects_scrubbed":16,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:26.900186+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0030839299999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4004,"num_read_kb":17323,"num_write":255,"num_write_kb":4995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":300331,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"148'345","reported_seq":5004,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.903958+0000","last_change":"2024-04-23T23:52:20.903958+0000","last_active":"2024-04-23T23:52:20.903958+0000","last_peered":"2024-04-23T23:52:20.903958+0000","last_clean":"2024-04-23T23:52:20.903958+0000","last_became_active":"2024-04-23T23:51:57.042413+0000","last_became_peered":"2024-04-23T23:51:57.042413+0000","last_unstale":"2024-04-23T23:52:20.903958+0000","last_undegraded":"2024-04-23T23:52:20.903958+0000","last_fullsized":"2024-04-23T23:52:20.903958+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'345","last_scrub_stamp":"2024-04-23T23:52:20.903838+0000","last_deep_scrub":"148'345","last_deep_scrub_stamp":"2024-04-23T23:52:20.903838+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.903838+0000","objects_scrubbed":12,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:52.381179+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002717367,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":4904,"num_read_kb":10560,"num_write":312,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":33204,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'803","reported_seq":6512,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:21.916702+0000","last_change":"2024-04-23T23:52:21.916748+0000","last_active":"2024-04-23T23:52:21.916702+0000","last_peered":"2024-04-23T23:52:21.916702+0000","last_clean":"2024-04-23T23:52:21.916702+0000","last_became_active":"2024-04-23T23:51:57.042909+0000","last_became_peered":"2024-04-23T23:51:57.042909+0000","last_unstale":"2024-04-23T23:52:21.916702+0000","last_undegraded":"2024-04-23T23:52:21.916702+0000","last_fullsized":"2024-04-23T23:52:21.916702+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'803","last_scrub_stamp":"2024-04-23T23:52:21.916598+0000","last_deep_scrub":"148'803","last_deep_scrub_stamp":"2024-04-23T23:52:21.916598+0000","last_clean_scrub_stamp":"2024-04-23T23:52:21.916598+0000","objects_scrubbed":15,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:45.342385+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00178959,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8124,"num_read_kb":18825,"num_write":1073,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":371235,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"148'634","reported_seq":3581,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:23.242538+0000","last_change":"2024-04-23T23:52:23.242574+0000","last_active":"2024-04-23T23:52:23.242538+0000","last_peered":"2024-04-23T23:52:23.242538+0000","last_clean":"2024-04-23T23:52:23.242538+0000","last_became_active":"2024-04-23T23:51:51.010820+0000","last_became_peered":"2024-04-23T23:51:51.010820+0000","last_unstale":"2024-04-23T23:52:23.242538+0000","last_undegraded":"2024-04-23T23:52:23.242538+0000","last_fullsized":"2024-04-23T23:52:23.242538+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'634","last_scrub_stamp":"2024-04-23T23:52:23.242455+0000","last_deep_scrub":"148'634","last_deep_scrub_stamp":"2024-04-23T23:52:23.242455+0000","last_clean_scrub_stamp":"2024-04-23T23:52:23.242455+0000","objects_scrubbed":8,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:05:52.476070+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0021784930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3962,"num_read_kb":16769,"num_write":805,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":1611508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'640","reported_seq":6396,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:18.794413+0000","last_change":"2024-04-23T23:52:18.794510+0000","last_active":"2024-04-23T23:52:18.794413+0000","last_peered":"2024-04-23T23:52:18.794413+0000","last_clean":"2024-04-23T23:52:18.794413+0000","last_became_active":"2024-04-23T23:51:51.011527+0000","last_became_peered":"2024-04-23T23:51:51.011527+0000","last_unstale":"2024-04-23T23:52:18.794413+0000","last_undegraded":"2024-04-23T23:52:18.794413+0000","last_fullsized":"2024-04-23T23:52:18.794413+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'640","last_scrub_stamp":"2024-04-23T23:52:18.794296+0000","last_deep_scrub":"148'640","last_deep_scrub_stamp":"2024-04-23T23:52:18.794296+0000","last_clean_scrub_stamp":"2024-04-23T23:52:18.794296+0000","objects_scrubbed":20,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:35.518439+0000","scrub_duration":12,"objects_trimmed":2,"snaptrim_duration":0.0079207389999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7310,"num_read_kb":34366,"num_write":713,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":422195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'477","reported_seq":6542,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:21.432942+0000","last_change":"2024-04-23T23:52:21.433011+0000","last_active":"2024-04-23T23:52:21.432942+0000","last_peered":"2024-04-23T23:52:21.432942+0000","last_clean":"2024-04-23T23:52:21.432942+0000","last_became_active":"2024-04-23T23:49:51.317584+0000","last_became_peered":"2024-04-23T23:49:51.317584+0000","last_unstale":"2024-04-23T23:52:21.432942+0000","last_undegraded":"2024-04-23T23:52:21.432942+0000","last_fullsized":"2024-04-23T23:52:21.432942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'477","last_scrub_stamp":"2024-04-23T23:52:21.432839+0000","last_deep_scrub":"148'477","last_deep_scrub_stamp":"2024-04-23T23:52:21.432839+0000","last_clean_scrub_stamp":"2024-04-23T23:52:21.432839+0000","objects_scrubbed":9,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:14:53.410637+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0023537699999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7428,"num_read_kb":40464,"num_write":381,"num_write_kb":20105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"148'377","reported_seq":2186,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:25.016061+0000","last_change":"2024-04-23T23:52:25.016134+0000","last_active":"2024-04-23T23:52:25.016061+0000","last_peered":"2024-04-23T23:52:25.016061+0000","last_clean":"2024-04-23T23:52:25.016061+0000","last_became_active":"2024-04-23T23:50:32.957938+0000","last_became_peered":"2024-04-23T23:50:32.957938+0000","last_unstale":"2024-04-23T23:52:25.016061+0000","last_undegraded":"2024-04-23T23:52:25.016061+0000","last_fullsized":"2024-04-23T23:52:25.016061+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'377","last_scrub_stamp":"2024-04-23T23:52:25.015962+0000","last_deep_scrub":"148'377","last_deep_scrub_stamp":"2024-04-23T23:52:25.015962+0000","last_clean_scrub_stamp":"2024-04-23T23:52:25.015962+0000","objects_scrubbed":11,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.744708+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0026696480000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1014,"num_read_kb":13100,"num_write":334,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":797880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"148'287","reported_seq":4579,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.924150+0000","last_change":"2024-04-23T23:52:19.924201+0000","last_active":"2024-04-23T23:52:19.924150+0000","last_peered":"2024-04-23T23:52:19.924150+0000","last_clean":"2024-04-23T23:52:19.924150+0000","last_became_active":"2024-04-23T23:51:57.040770+0000","last_became_peered":"2024-04-23T23:51:57.040770+0000","last_unstale":"2024-04-23T23:52:19.924150+0000","last_undegraded":"2024-04-23T23:52:19.924150+0000","last_fullsized":"2024-04-23T23:52:19.924150+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'287","last_scrub_stamp":"2024-04-23T23:52:19.924055+0000","last_deep_scrub":"148'287","last_deep_scrub_stamp":"2024-04-23T23:52:19.924055+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.924055+0000","objects_scrubbed":10,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:58:53.797605+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025361250000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3735,"num_read_kb":20798,"num_write":249,"num_write_kb":17430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":7180839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":274,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.817377+0000","last_change":"2024-04-23T23:52:19.817400+0000","last_active":"2024-04-23T23:52:19.817377+0000","last_peered":"2024-04-23T23:52:19.817377+0000","last_clean":"2024-04-23T23:52:19.817377+0000","last_became_active":"2024-04-23T23:49:43.259598+0000","last_became_peered":"2024-04-23T23:49:43.259598+0000","last_unstale":"2024-04-23T23:52:19.817377+0000","last_undegraded":"2024-04-23T23:52:19.817377+0000","last_fullsized":"2024-04-23T23:52:19.817377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:19.817288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:19.817288+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.817288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:49.253329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"148'877","reported_seq":3114,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:21.788822+0000","last_change":"2024-04-23T23:52:21.788863+0000","last_active":"2024-04-23T23:52:21.788822+0000","last_peered":"2024-04-23T23:52:21.788822+0000","last_clean":"2024-04-23T23:52:21.788822+0000","last_became_active":"2024-04-23T23:51:57.041069+0000","last_became_peered":"2024-04-23T23:51:57.041069+0000","last_unstale":"2024-04-23T23:52:21.788822+0000","last_undegraded":"2024-04-23T23:52:21.788822+0000","last_fullsized":"2024-04-23T23:52:21.788822+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'877","last_scrub_stamp":"2024-04-23T23:52:21.788739+0000","last_deep_scrub":"148'877","last_deep_scrub_stamp":"2024-04-23T23:52:21.788739+0000","last_clean_scrub_stamp":"2024-04-23T23:52:21.788739+0000","objects_scrubbed":13,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:17:09.011317+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0035059449999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4402,"num_read_kb":10655,"num_write":1166,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":680918,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:18.886136+0000","last_change":"2024-04-23T23:52:18.886172+0000","last_active":"2024-04-23T23:52:18.886136+0000","last_peered":"2024-04-23T23:52:18.886136+0000","last_clean":"2024-04-23T23:52:18.886136+0000","last_became_active":"2024-04-23T23:51:53.964251+0000","last_became_peered":"2024-04-23T23:51:53.964251+0000","last_unstale":"2024-04-23T23:52:18.886136+0000","last_undegraded":"2024-04-23T23:52:18.886136+0000","last_fullsized":"2024-04-23T23:52:18.886136+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:18.886011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:18.886011+0000","last_clean_scrub_stamp":"2024-04-23T23:52:18.886011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:45.113222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"148'635","reported_seq":4659,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:21.276673+0000","last_change":"2024-04-23T23:52:21.276703+0000","last_active":"2024-04-23T23:52:21.276673+0000","last_peered":"2024-04-23T23:52:21.276673+0000","last_clean":"2024-04-23T23:52:21.276673+0000","last_became_active":"2024-04-23T23:50:36.143959+0000","last_became_peered":"2024-04-23T23:50:36.143959+0000","last_unstale":"2024-04-23T23:52:21.276673+0000","last_undegraded":"2024-04-23T23:52:21.276673+0000","last_fullsized":"2024-04-23T23:52:21.276673+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'635","last_scrub_stamp":"2024-04-23T23:52:21.276589+0000","last_deep_scrub":"148'635","last_deep_scrub_stamp":"2024-04-23T23:52:21.276589+0000","last_clean_scrub_stamp":"2024-04-23T23:52:21.276589+0000","objects_scrubbed":11,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:28:57.892264+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001474712,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6670,"num_read_kb":16587,"num_write":647,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1320178,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":284,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:26.003672+0000","last_change":"2024-04-23T23:52:26.003694+0000","last_active":"2024-04-23T23:52:26.003672+0000","last_peered":"2024-04-23T23:52:26.003672+0000","last_clean":"2024-04-23T23:52:26.003672+0000","last_became_active":"2024-04-23T23:51:47.965875+0000","last_became_peered":"2024-04-23T23:51:47.965875+0000","last_unstale":"2024-04-23T23:52:26.003672+0000","last_undegraded":"2024-04-23T23:52:26.003672+0000","last_fullsized":"2024-04-23T23:52:26.003672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:26.003580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:26.003580+0000","last_clean_scrub_stamp":"2024-04-23T23:52:26.003580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:31:46.699150+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'474","reported_seq":2697,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.897972+0000","last_change":"2024-04-23T23:52:20.898016+0000","last_active":"2024-04-23T23:52:20.897972+0000","last_peered":"2024-04-23T23:52:20.897972+0000","last_clean":"2024-04-23T23:52:20.897972+0000","last_became_active":"2024-04-23T23:51:57.041394+0000","last_became_peered":"2024-04-23T23:51:57.041394+0000","last_unstale":"2024-04-23T23:52:20.897972+0000","last_undegraded":"2024-04-23T23:52:20.897972+0000","last_fullsized":"2024-04-23T23:52:20.897972+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'474","last_scrub_stamp":"2024-04-23T23:52:20.897894+0000","last_deep_scrub":"148'474","last_deep_scrub_stamp":"2024-04-23T23:52:20.897894+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.897894+0000","objects_scrubbed":14,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:38:47.090614+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00050010099999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3734,"num_read_kb":9257,"num_write":388,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":658843,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:17.860256+0000","last_change":"2024-04-23T23:52:17.860274+0000","last_active":"2024-04-23T23:52:17.860256+0000","last_peered":"2024-04-23T23:52:17.860256+0000","last_clean":"2024-04-23T23:52:17.860256+0000","last_became_active":"2024-04-23T23:51:53.964223+0000","last_became_peered":"2024-04-23T23:51:53.964223+0000","last_unstale":"2024-04-23T23:52:17.860256+0000","last_undegraded":"2024-04-23T23:52:17.860256+0000","last_fullsized":"2024-04-23T23:52:17.860256+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:17.860181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:17.860181+0000","last_clean_scrub_stamp":"2024-04-23T23:52:17.860181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:56:02.459020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"148'551","reported_seq":9079,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:23.483509+0000","last_change":"2024-04-23T23:52:23.483564+0000","last_active":"2024-04-23T23:52:23.483509+0000","last_peered":"2024-04-23T23:52:23.483509+0000","last_clean":"2024-04-23T23:52:23.483509+0000","last_became_active":"2024-04-23T23:50:36.143194+0000","last_became_peered":"2024-04-23T23:50:36.143194+0000","last_unstale":"2024-04-23T23:52:23.483509+0000","last_undegraded":"2024-04-23T23:52:23.483509+0000","last_fullsized":"2024-04-23T23:52:23.483509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'551","last_scrub_stamp":"2024-04-23T23:52:23.483421+0000","last_deep_scrub":"148'551","last_deep_scrub_stamp":"2024-04-23T23:52:23.483421+0000","last_clean_scrub_stamp":"2024-04-23T23:52:23.483421+0000","objects_scrubbed":10,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:46:10.433494+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.003109129,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":9448,"num_read_kb":39460,"num_write":576,"num_write_kb":11273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":658592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"148'753","reported_seq":5055,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.874169+0000","last_change":"2024-04-23T23:52:19.874215+0000","last_active":"2024-04-23T23:52:19.874169+0000","last_peered":"2024-04-23T23:52:19.874169+0000","last_clean":"2024-04-23T23:52:19.874169+0000","last_became_active":"2024-04-23T23:51:51.008961+0000","last_became_peered":"2024-04-23T23:51:51.008961+0000","last_unstale":"2024-04-23T23:52:19.874169+0000","last_undegraded":"2024-04-23T23:52:19.874169+0000","last_fullsized":"2024-04-23T23:52:19.874169+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'753","last_scrub_stamp":"2024-04-23T23:52:19.874076+0000","last_deep_scrub":"148'753","last_deep_scrub_stamp":"2024-04-23T23:52:19.874076+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.874076+0000","objects_scrubbed":12,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:30:41.789278+0000","scrub_duration":8,"objects_trimmed":2,"snaptrim_duration":0.0062357849999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6127,"num_read_kb":22953,"num_write":810,"num_write_kb":8079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":114406,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":284,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:26.529199+0000","last_change":"2024-04-23T23:52:26.529243+0000","last_active":"2024-04-23T23:52:26.529199+0000","last_peered":"2024-04-23T23:52:26.529199+0000","last_clean":"2024-04-23T23:52:26.529199+0000","last_became_active":"2024-04-23T23:51:53.964923+0000","last_became_peered":"2024-04-23T23:51:53.964923+0000","last_unstale":"2024-04-23T23:52:26.529199+0000","last_undegraded":"2024-04-23T23:52:26.529199+0000","last_fullsized":"2024-04-23T23:52:26.529199+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:26.529090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:26.529090+0000","last_clean_scrub_stamp":"2024-04-23T23:52:26.529090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:34.543335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"148'333","reported_seq":2770,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:22.944385+0000","last_change":"2024-04-23T23:52:22.944435+0000","last_active":"2024-04-23T23:52:22.944385+0000","last_peered":"2024-04-23T23:52:22.944385+0000","last_clean":"2024-04-23T23:52:22.944385+0000","last_became_active":"2024-04-23T23:51:57.043420+0000","last_became_peered":"2024-04-23T23:51:57.043420+0000","last_unstale":"2024-04-23T23:52:22.944385+0000","last_undegraded":"2024-04-23T23:52:22.944385+0000","last_fullsized":"2024-04-23T23:52:22.944385+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'333","last_scrub_stamp":"2024-04-23T23:52:22.944295+0000","last_deep_scrub":"148'333","last_deep_scrub_stamp":"2024-04-23T23:52:22.944295+0000","last_clean_scrub_stamp":"2024-04-23T23:52:22.944295+0000","objects_scrubbed":16,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:47:08.918501+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00086058499999999997,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2224,"num_read_kb":10117,"num_write":273,"num_write_kb":5989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":98175,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:18.665601+0000","last_change":"2024-04-23T23:52:18.665621+0000","last_active":"2024-04-23T23:52:18.665601+0000","last_peered":"2024-04-23T23:52:18.665601+0000","last_clean":"2024-04-23T23:52:18.665601+0000","last_became_active":"2024-04-23T23:50:32.953323+0000","last_became_peered":"2024-04-23T23:50:32.953323+0000","last_unstale":"2024-04-23T23:52:18.665601+0000","last_undegraded":"2024-04-23T23:52:18.665601+0000","last_fullsized":"2024-04-23T23:52:18.665601+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:18.665510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:18.665510+0000","last_clean_scrub_stamp":"2024-04-23T23:52:18.665510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:19.098490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"148'368","reported_seq":3203,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:17.812585+0000","last_change":"2024-04-23T23:52:17.812626+0000","last_active":"2024-04-23T23:52:17.812585+0000","last_peered":"2024-04-23T23:52:17.812585+0000","last_clean":"2024-04-23T23:52:17.812585+0000","last_became_active":"2024-04-23T23:51:51.008736+0000","last_became_peered":"2024-04-23T23:51:51.008736+0000","last_unstale":"2024-04-23T23:52:17.812585+0000","last_undegraded":"2024-04-23T23:52:17.812585+0000","last_fullsized":"2024-04-23T23:52:17.812585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'368","last_scrub_stamp":"2024-04-23T23:52:17.812511+0000","last_deep_scrub":"148'368","last_deep_scrub_stamp":"2024-04-23T23:52:17.812511+0000","last_clean_scrub_stamp":"2024-04-23T23:52:17.812511+0000","objects_scrubbed":15,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:13:32.309533+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0027204299999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3536,"num_read_kb":15405,"num_write":274,"num_write_kb":2565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":294,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:28.046316+0000","last_change":"2024-04-23T23:52:28.046363+0000","last_active":"2024-04-23T23:52:28.046316+0000","last_peered":"2024-04-23T23:52:28.046316+0000","last_clean":"2024-04-23T23:52:28.046316+0000","last_became_active":"2024-04-23T23:51:53.964154+0000","last_became_peered":"2024-04-23T23:51:53.964154+0000","last_unstale":"2024-04-23T23:52:28.046316+0000","last_undegraded":"2024-04-23T23:52:28.046316+0000","last_fullsized":"2024-04-23T23:52:28.046316+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:28.046212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:28.046212+0000","last_clean_scrub_stamp":"2024-04-23T23:52:28.046212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:18.952390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'643","reported_seq":2047,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.233444+0000","last_change":"2024-04-23T23:52:19.233444+0000","last_active":"2024-04-23T23:52:19.233444+0000","last_peered":"2024-04-23T23:52:19.233444+0000","last_clean":"2024-04-23T23:52:19.233444+0000","last_became_active":"2024-04-23T23:50:32.953999+0000","last_became_peered":"2024-04-23T23:50:32.953999+0000","last_unstale":"2024-04-23T23:52:19.233444+0000","last_undegraded":"2024-04-23T23:52:19.233444+0000","last_fullsized":"2024-04-23T23:52:19.233444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'643","last_scrub_stamp":"2024-04-23T23:52:19.233332+0000","last_deep_scrub":"148'643","last_deep_scrub_stamp":"2024-04-23T23:52:19.233332+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.233332+0000","objects_scrubbed":13,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:51.508701+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0017538779999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5787,"num_read_kb":22354,"num_write":803,"num_write_kb":14605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2807499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":274,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.434814+0000","last_change":"2024-04-23T23:52:19.434880+0000","last_active":"2024-04-23T23:52:19.434814+0000","last_peered":"2024-04-23T23:52:19.434814+0000","last_clean":"2024-04-23T23:52:19.434814+0000","last_became_active":"2024-04-23T23:49:43.264133+0000","last_became_peered":"2024-04-23T23:49:43.264133+0000","last_unstale":"2024-04-23T23:52:19.434814+0000","last_undegraded":"2024-04-23T23:52:19.434814+0000","last_fullsized":"2024-04-23T23:52:19.434814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T23:52:19.434701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T23:52:19.434701+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.434701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:21:19.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"148'119","reported_seq":458,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:27.054079+0000","last_change":"2024-04-23T23:52:27.054115+0000","last_active":"2024-04-23T23:52:27.054079+0000","last_peered":"2024-04-23T23:52:27.054079+0000","last_clean":"2024-04-23T23:52:27.054079+0000","last_became_active":"2024-04-23T23:51:51.009028+0000","last_became_peered":"2024-04-23T23:51:51.009028+0000","last_unstale":"2024-04-23T23:52:27.054079+0000","last_undegraded":"2024-04-23T23:52:27.054079+0000","last_fullsized":"2024-04-23T23:52:27.054079+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'119","last_scrub_stamp":"2024-04-23T23:52:27.053980+0000","last_deep_scrub":"148'119","last_deep_scrub_stamp":"2024-04-23T23:52:27.053980+0000","last_clean_scrub_stamp":"2024-04-23T23:52:27.053980+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:57:11.333881+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"148'333","reported_seq":1943,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:24.285039+0000","last_change":"2024-04-23T23:52:24.285085+0000","last_active":"2024-04-23T23:52:24.285039+0000","last_peered":"2024-04-23T23:52:24.285039+0000","last_clean":"2024-04-23T23:52:24.285039+0000","last_became_active":"2024-04-23T23:51:57.040273+0000","last_became_peered":"2024-04-23T23:51:57.040273+0000","last_unstale":"2024-04-23T23:52:24.285039+0000","last_undegraded":"2024-04-23T23:52:24.285039+0000","last_fullsized":"2024-04-23T23:52:24.285039+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'333","last_scrub_stamp":"2024-04-23T23:52:24.284944+0000","last_deep_scrub":"148'333","last_deep_scrub_stamp":"2024-04-23T23:52:24.284944+0000","last_clean_scrub_stamp":"2024-04-23T23:52:24.284944+0000","objects_scrubbed":12,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:44:10.238435+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001579261,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4225,"num_read_kb":8443,"num_write":287,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":106866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'535","reported_seq":2923,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:18.897604+0000","last_change":"2024-04-23T23:52:18.897690+0000","last_active":"2024-04-23T23:52:18.897604+0000","last_peered":"2024-04-23T23:52:18.897604+0000","last_clean":"2024-04-23T23:52:18.897604+0000","last_became_active":"2024-04-23T23:49:51.320205+0000","last_became_peered":"2024-04-23T23:49:51.320205+0000","last_unstale":"2024-04-23T23:52:18.897604+0000","last_undegraded":"2024-04-23T23:52:18.897604+0000","last_fullsized":"2024-04-23T23:52:18.897604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'535","last_scrub_stamp":"2024-04-23T23:52:18.897476+0000","last_deep_scrub":"148'535","last_deep_scrub_stamp":"2024-04-23T23:52:18.897476+0000","last_clean_scrub_stamp":"2024-04-23T23:52:18.897476+0000","objects_scrubbed":19,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:39:53.710220+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.002485105,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4310,"num_read_kb":12566,"num_write":423,"num_write_kb":8345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'458","reported_seq":3530,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.789451+0000","last_change":"2024-04-23T23:52:20.789496+0000","last_active":"2024-04-23T23:52:20.789451+0000","last_peered":"2024-04-23T23:52:20.789451+0000","last_clean":"2024-04-23T23:52:20.789451+0000","last_became_active":"2024-04-23T23:49:51.317933+0000","last_became_peered":"2024-04-23T23:49:51.317933+0000","last_unstale":"2024-04-23T23:52:20.789451+0000","last_undegraded":"2024-04-23T23:52:20.789451+0000","last_fullsized":"2024-04-23T23:52:20.789451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'458","last_scrub_stamp":"2024-04-23T23:52:20.789367+0000","last_deep_scrub":"148'458","last_deep_scrub_stamp":"2024-04-23T23:52:20.789367+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.789367+0000","objects_scrubbed":16,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:11:40.546738+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00094842600000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":2786,"num_read_kb":16929,"num_write":410,"num_write_kb":11497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"148'452","reported_seq":2970,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:22.281627+0000","last_change":"2024-04-23T23:52:22.281670+0000","last_active":"2024-04-23T23:52:22.281627+0000","last_peered":"2024-04-23T23:52:22.281627+0000","last_clean":"2024-04-23T23:52:22.281627+0000","last_became_active":"2024-04-23T23:51:51.009921+0000","last_became_peered":"2024-04-23T23:51:51.009921+0000","last_unstale":"2024-04-23T23:52:22.281627+0000","last_undegraded":"2024-04-23T23:52:22.281627+0000","last_fullsized":"2024-04-23T23:52:22.281627+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'452","last_scrub_stamp":"2024-04-23T23:52:22.281542+0000","last_deep_scrub":"148'452","last_deep_scrub_stamp":"2024-04-23T23:52:22.281542+0000","last_clean_scrub_stamp":"2024-04-23T23:52:22.281542+0000","objects_scrubbed":9,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:32.954558+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0048781299999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2844,"num_read_kb":21685,"num_write":459,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":3287152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"148'459","reported_seq":4098,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:22.466003+0000","last_change":"2024-04-23T23:52:22.466043+0000","last_active":"2024-04-23T23:52:22.466003+0000","last_peered":"2024-04-23T23:52:22.466003+0000","last_clean":"2024-04-23T23:52:22.466003+0000","last_became_active":"2024-04-23T23:50:36.143136+0000","last_became_peered":"2024-04-23T23:50:36.143136+0000","last_unstale":"2024-04-23T23:52:22.466003+0000","last_undegraded":"2024-04-23T23:52:22.466003+0000","last_fullsized":"2024-04-23T23:52:22.466003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'459","last_scrub_stamp":"2024-04-23T23:52:22.465925+0000","last_deep_scrub":"148'459","last_deep_scrub_stamp":"2024-04-23T23:52:22.465925+0000","last_clean_scrub_stamp":"2024-04-23T23:52:22.465925+0000","objects_scrubbed":16,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:39.629405+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0037268660000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3968,"num_read_kb":29146,"num_write":382,"num_write_kb":15937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1202900,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"148'372","reported_seq":3515,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:19.971325+0000","last_change":"2024-04-23T23:52:19.971365+0000","last_active":"2024-04-23T23:52:19.971325+0000","last_peered":"2024-04-23T23:52:19.971325+0000","last_clean":"2024-04-23T23:52:19.971325+0000","last_became_active":"2024-04-23T23:49:51.320583+0000","last_became_peered":"2024-04-23T23:49:51.320583+0000","last_unstale":"2024-04-23T23:52:19.971325+0000","last_undegraded":"2024-04-23T23:52:19.971325+0000","last_fullsized":"2024-04-23T23:52:19.971325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'372","last_scrub_stamp":"2024-04-23T23:52:19.971232+0000","last_deep_scrub":"148'372","last_deep_scrub_stamp":"2024-04-23T23:52:19.971232+0000","last_clean_scrub_stamp":"2024-04-23T23:52:19.971232+0000","objects_scrubbed":9,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:13.316999+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0033894099999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3559,"num_read_kb":24871,"num_write":324,"num_write_kb":14610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"148'596","reported_seq":3696,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:21.955177+0000","last_change":"2024-04-23T23:52:21.955215+0000","last_active":"2024-04-23T23:52:21.955177+0000","last_peered":"2024-04-23T23:52:21.955177+0000","last_clean":"2024-04-23T23:52:21.955177+0000","last_became_active":"2024-04-23T23:49:51.322779+0000","last_became_peered":"2024-04-23T23:49:51.322779+0000","last_unstale":"2024-04-23T23:52:21.955177+0000","last_undegraded":"2024-04-23T23:52:21.955177+0000","last_fullsized":"2024-04-23T23:52:21.955177+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'596","last_scrub_stamp":"2024-04-23T23:52:21.955093+0000","last_deep_scrub":"148'596","last_deep_scrub_stamp":"2024-04-23T23:52:21.955093+0000","last_clean_scrub_stamp":"2024-04-23T23:52:21.955093+0000","objects_scrubbed":10,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:55:58.363133+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026091069999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4543,"num_read_kb":16422,"num_write":714,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'874","reported_seq":4057,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:24.009106+0000","last_change":"2024-04-23T23:52:24.009154+0000","last_active":"2024-04-23T23:52:24.009106+0000","last_peered":"2024-04-23T23:52:24.009106+0000","last_clean":"2024-04-23T23:52:24.009106+0000","last_became_active":"2024-04-23T23:49:51.322829+0000","last_became_peered":"2024-04-23T23:49:51.322829+0000","last_unstale":"2024-04-23T23:52:24.009106+0000","last_undegraded":"2024-04-23T23:52:24.009106+0000","last_fullsized":"2024-04-23T23:52:24.009106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'874","last_scrub_stamp":"2024-04-23T23:52:24.009020+0000","last_deep_scrub":"148'874","last_deep_scrub_stamp":"2024-04-23T23:52:24.009020+0000","last_clean_scrub_stamp":"2024-04-23T23:52:24.009020+0000","objects_scrubbed":16,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:14.158487+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.002637707,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5727,"num_read_kb":22935,"num_write":1222,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"148'336","reported_seq":5869,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:21.781472+0000","last_change":"2024-04-23T23:52:21.781546+0000","last_active":"2024-04-23T23:52:21.781472+0000","last_peered":"2024-04-23T23:52:21.781472+0000","last_clean":"2024-04-23T23:52:21.781472+0000","last_became_active":"2024-04-23T23:51:51.011061+0000","last_became_peered":"2024-04-23T23:51:51.011061+0000","last_unstale":"2024-04-23T23:52:21.781472+0000","last_undegraded":"2024-04-23T23:52:21.781472+0000","last_fullsized":"2024-04-23T23:52:21.781472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'336","last_scrub_stamp":"2024-04-23T23:52:21.781354+0000","last_deep_scrub":"148'336","last_deep_scrub_stamp":"2024-04-23T23:52:21.781354+0000","last_clean_scrub_stamp":"2024-04-23T23:52:21.781354+0000","objects_scrubbed":15,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:19.059787+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0011928069999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5438,"num_read_kb":16762,"num_write":237,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'422","reported_seq":4219,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:23.000948+0000","last_change":"2024-04-23T23:52:23.001002+0000","last_active":"2024-04-23T23:52:23.000948+0000","last_peered":"2024-04-23T23:52:23.000948+0000","last_clean":"2024-04-23T23:52:23.000948+0000","last_became_active":"2024-04-23T23:49:51.322915+0000","last_became_peered":"2024-04-23T23:49:51.322915+0000","last_unstale":"2024-04-23T23:52:23.000948+0000","last_undegraded":"2024-04-23T23:52:23.000948+0000","last_fullsized":"2024-04-23T23:52:23.000948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'422","last_scrub_stamp":"2024-04-23T23:52:23.000855+0000","last_deep_scrub":"148'422","last_deep_scrub_stamp":"2024-04-23T23:52:23.000855+0000","last_clean_scrub_stamp":"2024-04-23T23:52:23.000855+0000","objects_scrubbed":11,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:47:56.781161+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00067280200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4104,"num_read_kb":10536,"num_write":367,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"148'461","reported_seq":2366,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:25.506541+0000","last_change":"2024-04-23T23:52:25.506584+0000","last_active":"2024-04-23T23:52:25.506541+0000","last_peered":"2024-04-23T23:52:25.506541+0000","last_clean":"2024-04-23T23:52:25.506541+0000","last_became_active":"2024-04-23T23:51:38.964578+0000","last_became_peered":"2024-04-23T23:51:38.964578+0000","last_unstale":"2024-04-23T23:52:25.506541+0000","last_undegraded":"2024-04-23T23:52:25.506541+0000","last_fullsized":"2024-04-23T23:52:25.506541+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'461","last_scrub_stamp":"2024-04-23T23:52:25.506459+0000","last_deep_scrub":"148'461","last_deep_scrub_stamp":"2024-04-23T23:52:25.506459+0000","last_clean_scrub_stamp":"2024-04-23T23:52:25.506459+0000","objects_scrubbed":10,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:27:15.284114+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001265272,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2121,"num_read_kb":16796,"num_write":420,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1236255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"148'499","reported_seq":3090,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-23T23:52:20.942784+0000","last_change":"2024-04-23T23:52:20.942836+0000","last_active":"2024-04-23T23:52:20.942784+0000","last_peered":"2024-04-23T23:52:20.942784+0000","last_clean":"2024-04-23T23:52:20.942784+0000","last_became_active":"2024-04-23T23:49:51.322794+0000","last_became_peered":"2024-04-23T23:49:51.322794+0000","last_unstale":"2024-04-23T23:52:20.942784+0000","last_undegraded":"2024-04-23T23:52:20.942784+0000","last_fullsized":"2024-04-23T23:52:20.942784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'499","last_scrub_stamp":"2024-04-23T23:52:20.942696+0000","last_deep_scrub":"148'499","last_deep_scrub_stamp":"2024-04-23T23:52:20.942696+0000","last_clean_scrub_stamp":"2024-04-23T23:52:20.942696+0000","objects_scrubbed":13,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:45:32.294943+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0020507170000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4084,"num_read_kb":15005,"num_write":410,"num_write_kb":16101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":403,"num_read":156443,"num_read_kb":614208,"num_write":17651,"num_write_kb":303123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3125,"num_bytes_recovered":27981403,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},"log_size":16975,"ondisk_log_size":16975,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":56,"seq":240518168603,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46920,"kb_used_data":1728,"kb_used_omap":10,"kb_used_meta":45173,"kb_avail":94324920,"statfs":{"total":96636764160,"available":96588718080,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10998,"internal_metadata":46257418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50956,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320884,"statfs":{"total":96636764160,"available":96584585216,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55692,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316148,"statfs":{"total":96636764160,"available":96579735552,"internally_reserved":0,"allocated":2363392,"data_stored":1438944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37064,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334776,"statfs":{"total":96636764160,"available":96598810624,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50952,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320888,"statfs":{"total":96636764160,"available":96584589312,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1773568,"data_stored":848595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37064,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334776,"statfs":{"total":96636764160,"available":96598810624,"internally_reserved":0,"allocated":1769472,"data_stored":848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8211,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T23:52:40.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-23T23:52:40.865 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T23:52:40.865 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T23:52:40.865 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-23T23:52:40.865 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:40.908 INFO:tasks.ceph.osd.0:Stopped 2024-04-23T23:52:40.909 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-23T23:52:40.909 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:40.942 INFO:tasks.ceph.osd.1:Stopped 2024-04-23T23:52:40.942 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-23T23:52:40.942 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.012 INFO:tasks.ceph.osd.2:Stopped 2024-04-23T23:52:41.012 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-23T23:52:41.012 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.091 INFO:tasks.ceph.osd.3:Stopped 2024-04-23T23:52:41.091 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-23T23:52:41.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.133 INFO:tasks.ceph.osd.4:Stopped 2024-04-23T23:52:41.133 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-23T23:52:41.133 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.178 INFO:tasks.ceph.osd.5:Stopped 2024-04-23T23:52:41.178 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-23T23:52:41.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.224 INFO:tasks.ceph.osd.6:Stopped 2024-04-23T23:52:41.224 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-23T23:52:41.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.263 INFO:tasks.ceph.osd.7:Stopped 2024-04-23T23:52:41.263 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T23:52:41.263 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-23T23:52:41.263 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.287 INFO:tasks.ceph.mgr.y:Stopped 2024-04-23T23:52:41.287 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-23T23:52:41.287 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.331 INFO:tasks.ceph.mgr.x:Stopped 2024-04-23T23:52:41.331 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T23:52:41.331 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-23T23:52:41.331 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.342 INFO:tasks.ceph.mon.a:Stopped 2024-04-23T23:52:41.342 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-23T23:52:41.343 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.393 INFO:tasks.ceph.mon.c:Stopped 2024-04-23T23:52:41.394 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-23T23:52:41.394 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T23:52:41.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-23T23:52:41.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-23T23:52:41.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-23T23:52:41.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-23T23:52:41.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-23T23:52:41.406 INFO:tasks.ceph.mon.b:Stopped 2024-04-23T23:52:41.406 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T23:52:41.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-04-23T23:52:41.477 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:52:41.477 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-23T23:52:41.709 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:52:41.709 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-23T23:52:41.775 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:52:41.775 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-23T23:52:41.831 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:52:41.831 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-23T23:52:41.881 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:52:41.881 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-23T23:52:42.070 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:52:42.071 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-23T23:52:42.140 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:52:42.140 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-23T23:52:42.190 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:52:42.190 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-23T23:52:42.274 INFO:tasks.ceph:Archiving mon data... 2024-04-23T23:52:42.275 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/data/mon.a.tgz 2024-04-23T23:52:42.276 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-04-23T23:52:42.291 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.2u6AaQoMOg 2024-04-23T23:52:42.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2u6AaQoMOg 2024-04-23T23:52:42.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.2u6AaQoMOg 2024-04-23T23:52:42.518 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.2u6AaQoMOg is 366KB 2024-04-23T23:52:42.614 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.2u6AaQoMOg 2024-04-23T23:52:42.629 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/data/mon.c.tgz 2024-04-23T23:52:42.629 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-04-23T23:52:42.684 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.n7jxfjO0rf 2024-04-23T23:52:42.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.n7jxfjO0rf 2024-04-23T23:52:42.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.n7jxfjO0rf 2024-04-23T23:52:42.926 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.n7jxfjO0rf is 500KB 2024-04-23T23:52:43.024 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.n7jxfjO0rf 2024-04-23T23:52:43.039 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/data/mon.b.tgz 2024-04-23T23:52:43.039 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-04-23T23:52:43.057 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.eDatNElWpa 2024-04-23T23:52:43.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eDatNElWpa 2024-04-23T23:52:43.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.eDatNElWpa 2024-04-23T23:52:43.322 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.eDatNElWpa is 500KB 2024-04-23T23:52:43.419 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.eDatNElWpa 2024-04-23T23:52:43.434 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T23:52:43.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T23:52:43.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T23:52:43.512 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T23:52:43.551 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T23:52:43.551 INFO:tasks.ceph:Archiving crash dumps... 2024-04-23T23:52:43.553 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/remote/smithi092/crash 2024-04-23T23:52:43.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T23:52:43.577 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/remote/smithi119/crash 2024-04-23T23:52:43.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T23:52:43.613 INFO:tasks.ceph:Compressing logs... 2024-04-23T23:52:43.614 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T23:52:43.621 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T23:52:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58439.log 2024-04-23T23:52:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54815.log 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32082.log 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32082.log.gz 2024-04-23T23:52:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32162.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: 84.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32202.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32162.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-osd.4.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32162.log.gz 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32202.log.gzgzip 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32002.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32122.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32002.log.gz 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31922.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32122.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32122.log.gz 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31962.log 2024-04-23T23:52:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31922.log.gz 2024-04-23T23:52:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-23T23:52:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31962.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31962.log.gz 2024-04-23T23:52:43.701 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32042.log 2024-04-23T23:52:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.32042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32042.log.gz 2024-04-23T23:52:43.756 INFO:teuthology.orchestra.run.smithi119.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59107.log 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58439.log.gz 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.54815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57786.log 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54815.log.gz 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38896.log 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59107.log.gz 2024-04-23T23:52:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60871.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57786.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55365.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38896.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.60871.log: /var/log/ceph/ceph-client.admin.40739.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60871.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.55365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50063.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55365.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49721.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40739.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50063.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59919.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49721.log.gz 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40998.log 2024-04-23T23:52:43.777 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51793.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59919.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41587.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40998.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51793.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37510.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41587.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38770.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52649.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37510.log.gz 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55432.log 2024-04-23T23:52:43.778 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38770.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42422.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52649.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.55432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59527.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55432.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60605.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42422.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60248.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59527.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37810.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60605.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52404.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60248.log.gz 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61291.log 2024-04-23T23:52:43.779 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37810.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40092.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52404.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40289.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61291.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40719.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40092.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40289.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40289.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48658.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40719.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38131.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37808.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48658.log.gz 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49902.log 2024-04-23T23:52:43.780 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38131.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37808.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49902.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.49902.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48904.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59695.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48904.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60244.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59695.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38154.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59915.log 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60244.log.gz 2024-04-23T23:52:43.781 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48412.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47218.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59915.log.gz 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38728.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48412.log.gz 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.47218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46190.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47218.log.gz 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38728.log.gz 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57036.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56927.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46190.log.gz 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54199.log 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57036.log.gz 2024-04-23T23:52:43.782 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54149.log 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56927.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.54199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54199.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54573.log 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.54149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51286.log 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54149.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.54573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56394.log 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54573.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 0.0% 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51286.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61753.log 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56394.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46212.log 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2024-04-23T23:52:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60254.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61753.log.gz 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46212.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46212.log.gz 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48473.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60254.log.gz 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38133.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37814.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48473.log.gz 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38133.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38133.log.gz /var/log/ceph/ceph-client.admin.59275.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40820.log 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37814.log.gz 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59275.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.55835.log -- replaced with /var/log/ceph/ceph-client.admin.59275.log.gz 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52917.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40820.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.55835.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48816.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55835.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40957.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52917.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48816.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40957.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59443.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51875.log 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-04-23T23:52:43.785 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59443.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57642.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51875.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60829.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46879.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57642.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39562.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60829.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40659.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46879.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39562.log.gz 2024-04-23T23:52:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60787.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40659.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49862.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41410.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60787.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60598.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49862.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41410.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41410.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41822.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60598.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56640.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52812.log 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41822.log.gz 2024-04-23T23:52:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37636.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56640.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50023.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52812.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58961.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37636.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50023.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60955.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38113.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58961.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48949.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60955.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38113.log.gz 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59913.log 2024-04-23T23:52:43.788 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48949.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38487.log 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 0.0% 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59913.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38487.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60471.log 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38489.log 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60471.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60471.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61795.log 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38489.log: -- /var/log/ceph/ceph-client.admin.53370.log 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38489.log.gz 2024-04-23T23:52:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53780.log 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61795.log.gz 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49942.log 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53370.log.gz 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61501.log 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53780.log.gz 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61249.log 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49942.log.gz 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61501.log.gz 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45032.log 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40048.log 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61249.log.gz 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45032.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45032.log.gz -5 2024-04-23T23:52:43.790 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38488.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40048.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40048.log.gz 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38488.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-mgr.y.log -- replaced with /var/log/ceph/ceph-client.admin.38488.log.gz 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56770.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60115.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56770.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.44077.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56770.log.gz 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60115.log.gz 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.44077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46837.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32215.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-23T23:52:43.791 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46837.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58770.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32215.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39521.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32215.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58770.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38134.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56578.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38134.log: /var/log/ceph/ceph-client.admin.39521.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39521.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38134.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39000.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38485.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56578.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39000.log.gz 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49781.log 2024-04-23T23:52:43.792 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38485.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42421.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38485.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37323.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49781.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5/var/log/ceph/ceph-client.admin.42421.log: --verbose -- /var/log/ceph/ceph-client.admin.51659.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42421.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37323.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58592.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51659.log: /var/log/ceph/ceph-client.admin.56141.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51659.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58592.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52364.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.44077.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54444.log 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56141.log.gz 2024-04-23T23:52:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61081.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52364.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.54444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50389.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54444.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61081.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61081.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61627.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50389.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54791.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61627.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61627.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.54791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54791.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.45098.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40469.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/ceph-mon.c.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.45098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60243.log 2024-04-23T23:52:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49539.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60243.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59653.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60243.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49539.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40218.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59653.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40218.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45944.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38130.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58874.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45944.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38130.log.gz 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39139.log 2024-04-23T23:52:43.795 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58874.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61039.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39139.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61039.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60913.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.45098.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56329.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60913.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61417.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60913.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56329.log.gz 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49717.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-04-23T23:52:43.796 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61417.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49717.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32517.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37804.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32517.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37804.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60597.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.49983.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43211.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60597.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51032.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.43211.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.43211.log.gz 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60997.log 2024-04-23T23:52:43.797 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59485.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51032.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60997.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41757.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59485.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59191.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39379.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41757.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59191.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58380.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39379.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52532.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40876.log 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58380.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52532.log.gz 2024-04-23T23:52:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52142.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48179.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40876.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52142.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60253.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48179.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39950.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61375.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60253.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39950.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51033.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53259.log 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61375.log.gz 2024-04-23T23:52:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51033.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59233.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61333.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53259.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59233.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60451.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61333.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59737.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55725.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60451.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59737.log.gz 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57295.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.55725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40557.log 2024-04-23T23:52:43.800 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55725.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57295.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43817.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40114.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40557.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43817.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38115.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52145.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40114.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38115.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60600.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56239.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52145.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60600.log.gz 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41951.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51957.log 2024-04-23T23:52:43.801 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56239.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59912.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51518.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51957.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59912.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59917.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51518.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60255.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51517.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59917.log.gz 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60255.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58790.log 2024-04-23T23:52:43.802 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60255.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53005.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51517.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58790.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50516.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61669.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53005.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50516.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38350.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59149.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61669.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38350.log.gz 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47134.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57190.log 2024-04-23T23:52:43.803 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59149.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.47134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47134.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53721.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37815.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57190.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53721.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50781.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57581.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37815.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50781.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60599.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61585.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57581.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60599.log.gz 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53193.log 2024-04-23T23:52:43.804 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59317.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61585.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53193.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37818.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59317.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58081.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58894.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58081.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39640.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61711.log 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58894.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39640.log.gz 2024-04-23T23:52:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56515.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61711.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26352.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61711.log.gz 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.56515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56515.log.gz 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52039.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.tmp-client.admin.26352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26352.log.gz 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52039.log.gz 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61543.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.53107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46540.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61543.log.gz 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45879.log 2024-04-23T23:52:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61837.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45879.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45879.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.59914.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51584.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61837.log.gz 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59914.log.gz 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61123.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.51584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51584.log.gz 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57251.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61123.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59918.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61123.log.gz 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.57251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57251.log.gz 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50640.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59918.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59918.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-04-23T23:52:43.807 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52323.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50640.log: /var/log/ceph/ceph-client.admin.37468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50640.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59569.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52323.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59916.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52323.log.gz 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59359.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59569.log: /var/log/ceph/ceph-client.admin.59916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59569.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59916.log.gz 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37813.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59359.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52038.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59359.log.gz 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59611.log 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37813.log: /var/log/ceph/ceph-client.admin.52038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37813.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52038.log.gz 2024-04-23T23:52:43.808 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38812.log 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59611.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39052.log 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59611.log.gz 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32636.log 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38812.log: /var/log/ceph/ceph-client.0.39052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38812.log.gz 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50910.log 95.1% -- replaced with /var/log/ceph/ceph-client.0.39052.log.gz 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57991.log 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32636.log.gz 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50910.log.gz 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38938.log 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42191.log 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38938.log: /var/log/ceph/ceph-client.admin.57991.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57991.log.gz 2024-04-23T23:52:43.809 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38938.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55010.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.42191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41332.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.55010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55010.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40718.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41332.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40619.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40718.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61877.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40619.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61165.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61877.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48107.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61165.log.gz 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58017.log 2024-04-23T23:52:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.48107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48107.log.gz 95.0% -- replaced with /var/log/ceph/objectstore_tool.42191.log.gz 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52283.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.58017.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58017.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.51406.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37552.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.52283.log: /var/log/ceph/ceph-client.admin.51406.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52283.log.gz 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60245.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37552.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59401.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37552.log.gz 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37412.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.60245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60245.log.gz 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59401.log.gz 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49289.log 2024-04-23T23:52:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37412.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38686.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59025.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38686.log: /var/log/ceph/ceph-client.admin.49289.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38686.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49289.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39620.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.59025.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37594.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59025.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39620.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37594.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.38483.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37594.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46674.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38483.log.gz 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55111.log 2024-04-23T23:52:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55561.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46674.log: /var/log/ceph/ceph-client.admin.55111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46674.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55111.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39809.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55875.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.55561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55561.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39809.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38132.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37807.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38132.log: /var/log/ceph/ceph-client.admin.55875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55875.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38132.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50515.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61207.log 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37807.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.50515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50515.log.gz 2024-04-23T23:52:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.61207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61207.log.gz 2024-04-23T23:52:43.821 INFO:teuthology.orchestra.run.smithi092.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-23T23:52:43.826 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-23T23:52:44.006 INFO:teuthology.orchestra.run.smithi092.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-23T23:52:44.058 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-23T23:52:44.797 INFO:teuthology.orchestra.run.smithi119.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-23T23:52:45.315 INFO:teuthology.orchestra.run.smithi092.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-23T23:52:45.587 INFO:teuthology.orchestra.run.smithi092.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-23T23:52:45.899 INFO:teuthology.orchestra.run.smithi119.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-23T23:52:46.173 INFO:teuthology.orchestra.run.smithi119.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-23T23:52:46.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-23T23:52:46.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-23T23:52:46.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-23T23:52:46.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-23T23:52:46.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-23T23:52:46.760 INFO:teuthology.orchestra.run.smithi092.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-23T23:52:46.762 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-23T23:52:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m3.130s 2024-04-23T23:52:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m7.332s 2024-04-23T23:52:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.430s 2024-04-23T23:52:46.958 INFO:teuthology.orchestra.run.smithi119.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-23T23:52:46.959 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-23T23:52:46.959 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m3.284s 2024-04-23T23:52:46.960 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m8.937s 2024-04-23T23:52:46.960 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.368s 2024-04-23T23:52:46.960 INFO:tasks.ceph:Archiving logs... 2024-04-23T23:52:46.960 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/remote/smithi092/log 2024-04-23T23:52:46.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T23:52:48.079 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/remote/smithi119/log 2024-04-23T23:52:48.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T23:52:49.238 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T23:52:49.247 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T23:52:49.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T23:52:49.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T23:52:49.328 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T23:52:49.337 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T23:52:49.337 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T23:52:49.340 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T23:52:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-23T23:52:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-04-23T23:52:49.358 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T23:52:49.358 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-23T23:52:49.358 INFO:teuthology.orchestra.run.smithi092.stdout:^- hv01.front.sepia.ceph.com 2 6 377 9 +1759us[+1757us] +/- 591ms 2024-04-23T23:52:49.359 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -358us[ -360us] +/- 60ms 2024-04-23T23:52:49.359 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +507us[ +505us] +/- 84ms 2024-04-23T23:52:49.359 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:52:49.361 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T23:52:49.361 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-04-23T23:52:49.361 INFO:teuthology.orchestra.run.smithi119.stdout:^- hv01.front.sepia.ceph.com 2 6 377 8 +1766us[+1766us] +/- 591ms 2024-04-23T23:52:49.361 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -352us[ -350us] +/- 60ms 2024-04-23T23:52:49.362 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +515us[ +517us] +/- 84ms 2024-04-23T23:52:49.362 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T23:52:49.363 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T23:52:49.373 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T23:52:49.374 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T23:52:49.384 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T23:52:49.475 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T23:52:49.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T23:52:49.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:52:49.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T23:52:49.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:52:49.607 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T23:52:49.620 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T23:52:49.630 INFO:teuthology.task.internal:Duration was 617.295286 seconds 2024-04-23T23:52:49.630 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T23:52:49.639 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T23:52:49.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T23:52:49.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T23:52:49.676 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T23:52:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T23:52:50.078 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T23:52:50.078 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-04-23T23:52:50.079 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T23:52:50.139 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-04-23T23:52:50.139 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T23:52:50.173 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T23:52:50.174 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T23:52:50.181 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T23:52:50.241 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T23:52:50.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T23:52:50.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T23:52:50.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T23:52:50.407 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T23:52:50.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T23:52:50.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T23:52:50.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T23:52:50.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T23:52:50.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T23:52:50.493 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-04-23T23:52:50.517 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-04-23T23:52:50.561 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T23:52:50.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:52:50.575 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T23:52:50.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T23:52:50.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T23:52:50.625 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T23:52:50.626 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/remote/smithi092 2024-04-23T23:52:50.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T23:52:50.666 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-23_21:16:02-rbd-squid-distro-default-smithi/7670490/remote/smithi119 2024-04-23T23:52:50.667 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T23:52:50.698 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T23:52:50.698 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T23:52:50.701 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T23:52:50.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T23:52:50.760 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T23:52:50.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T23:52:50.770 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T23:52:50.770 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T23:52:50.773 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T23:52:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 23:52 /home/ubuntu/cephtest 2024-04-23T23:52:50.802 INFO:teuthology.orchestra.run.smithi119.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 23:52 /home/ubuntu/cephtest 2024-04-23T23:52:50.803 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T23:52:50.814 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T23:52:50.839 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} duration: 617.2952857017517 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-23T23:52:50.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T23:52:50.921 INFO:teuthology.run:pass